toga-ai 1.0.299 → 1.0.301
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
|
@@ -6,6 +6,6 @@
|
|
|
6
6
|
| [Compass MA Sales Order Exception Report](features/compass-ma-sales-order-exception-report.md) | A worker cron that emails operations the "Compass Refresh Exception Report" — Compass `MA%` sales orders whose corresponding Office Depot (ODP) sales order has | worker/crons/toga2/compass/workflow/7_generate_ma_sales_order_exception_report.php |
|
|
7
7
|
| [Compass Partial In-Transit & Delivered Emails (per package)](features/compass-partial-in-transit-delivered-emails.md) | Compass USA and Compass Canada send a **per-package** in-transit email (and a matching delivered email) instead of one email listing the whole order. | worker/crons/toga2/compass/update_salesorder_status_from_odp.php, worker/crons/toga2/compasscanada/workflow/3_update_salesorder_status_from_grand_and_toy.php |
|
|
8
8
|
| [Forecast2 ↔ NetSuite Reconciliation & Trueup Tooling](features/forecast2-netsuite-reconciliation.md) | CLI tools to **audit** and **repair** drift between the production `Forecast` DB (core2) and NetSuite. | test/@dave/checker.php, _underscore/Component/Forecast/SaleImport/SaleImport.php, test/@dave/looper.php, test/@dave/reconcile_netsuite_totals.php, test/@dave/fixer.php, test/@dave/analyze_netsuite_forecast_diff.php, test/@dave/trueup_sales.php, test/@dave/trueup_open_orders.php, test/@dave/loop_trueup_open_orders.php, test/@dave/trueup_opportunities.php, test/@dave/probe_sales_gap_direct.php, test/@dave/probe_missing_oo_timing.php, test/@dave/probe_missing_oo_createdby.php, test/@dave/probe_drift_so_dates.php, test/@dave/probe_profit_invoices.php, test/@dave/probe_profit_gap.php, worker/crons/toga2/forecast2/common_import_sales_from_netsuite.php, worker/crons/toga2/forecast2/periodic_forecast_discrepancy_fix_open_orders.php, worker/crons/toga2/forecast2/import_open_orders.php, worker/schedules/cron.worker.infrastructure.json |
|
|
9
|
-
| [NetSuite → TOGa Supply Per-Client Sync (thin wrappers)](features/netsuite-togasupply-per-client-sync.md) | Syncs NetSuite transactions (sales orders, purchase orders, invoices, item receipts, item fulfillments, inventory adjustments) into each TOGa Supply (2.0) clien | worker/crons/toga2/netsuite/common_sync_togasupply.php, worker/crons/toga2/netsuite/sync_togasupply_canon.php, worker/schedules/cron.worker.sync.json, dbchanges2/_modules/netsuite/2026-04-01 - Parameters.sql |
|
|
9
|
+
| [NetSuite → TOGa Supply Per-Client Sync (thin wrappers)](features/netsuite-togasupply-per-client-sync.md) | Syncs NetSuite transactions (sales orders, purchase orders, invoices, item receipts, item fulfillments, inventory adjustments) into each TOGa Supply (2.0) clien | worker/crons/toga2/netsuite/common_sync_togasupply.php, worker/crons/toga2/netsuite/sync_togasupply_canon.php, worker/schedules/cron.worker.sync.json, dbchanges2/_modules/netsuite/2026-04-01 - Parameters.sql, library/app/api/netsuite/rest.php, library/app/systemmonitor/netsuiteintegration.php |
|
|
10
10
|
| [Prudential: Send Shipments for the Day report (daily cron)](features/send-shipments-for-the-day.md) | Daily cron (9:00 PM) that emails Prudential and Dell stakeholders an Excel report of all devices shipped that day, including tracking number, serial number, emp | worker/crons/notifications/reports/send_shipments_for_the_day.php |
|
|
11
11
|
| [Onboarding a Client to the NetSuite TOGa Supply Sync](workflows/onboarding-client-to-netsuite-togasupply-sync.md) | How to add a new TOGa 2 client to the per-client NetSuite → TOGa Supply importer (`worker/crons/toga2/netsuite/`). | worker/crons/toga2/netsuite/sync_togasupply.php, worker/crons/toga2/netsuite/common_sync_togasupply.php, worker/schedules/cron.worker.sync.json, dbchanges2/_modules/netsuite/2026-04-01 - Parameters.sql |
|
|
@@ -6,13 +6,15 @@ project: Worker
|
|
|
6
6
|
client: shared
|
|
7
7
|
type: feature
|
|
8
8
|
status: active
|
|
9
|
-
updated: 2026-
|
|
9
|
+
updated: 2026-07-09
|
|
10
10
|
owners: ["dfranks", "bala"]
|
|
11
11
|
files:
|
|
12
12
|
- worker/crons/toga2/netsuite/common_sync_togasupply.php
|
|
13
13
|
- worker/crons/toga2/netsuite/sync_togasupply_canon.php
|
|
14
14
|
- worker/schedules/cron.worker.sync.json
|
|
15
15
|
- dbchanges2/_modules/netsuite/2026-04-01 - Parameters.sql
|
|
16
|
+
- library/app/api/netsuite/rest.php
|
|
17
|
+
- library/app/systemmonitor/netsuiteintegration.php
|
|
16
18
|
related:
|
|
17
19
|
- ../architecture.md
|
|
18
20
|
- forecast2-netsuite-reconciliation.md
|
|
@@ -128,9 +130,34 @@ Parameters are stored **per client DB** but accessed **through the TOGa2 API**,
|
|
|
128
130
|
older** integration (`syncWithToga`/`syncWithTogadesk`) with its own legacy parameter keys
|
|
129
131
|
(`NETSUITE_LAST_*_INTEGRATION_DATETIME`) — NOT the supply sync. The supply wrapper for AIG
|
|
130
132
|
lives in the `netsuite/` folder. Both key sets coexist harmlessly.
|
|
131
|
-
- **Invoices lag ~24h by design
|
|
132
|
-
(invoices may still be attaching), so `LAST_SYNC_DATETIME_INVOICES`
|
|
133
|
-
by a day. Not a fault.
|
|
133
|
+
- **Invoices lag ~24h by design + are keyed on `lastmodifieddate`:** the invoice section caps its
|
|
134
|
+
upper bound at `now − 86400s` (invoices may still be attaching), so `LAST_SYNC_DATETIME_INVOICES`
|
|
135
|
+
always trails the others by a day. Not a fault. Invoices are also pulled by
|
|
136
|
+
**`lastmodifieddate`, NOT `trandate`** — `App_Api_Netsuite_Rest::listSales()` defaults
|
|
137
|
+
`$dateField='lastmodifieddate'` and the engine calls it without the 4th arg. So a backdated
|
|
138
|
+
`trandate` is irrelevant to whether an invoice imports, and a just-modified invoice still won't
|
|
139
|
+
import until it is >24h old (the upper-bound cap).
|
|
140
|
+
- **A parse error in the shared `library/app/api/netsuite/rest.php` takes down EVERY client at
|
|
141
|
+
once (tier-wide outage).** All wrappers autoload `App_Api_Netsuite_Rest` in the pre-section
|
|
142
|
+
LOOKUPS/setup phase of `common_sync_togasupply.php` (~line 433). Because PHP parses the whole
|
|
143
|
+
file before executing, any fatal parse error in `rest.php` prevents the class from loading and
|
|
144
|
+
crashes **all** clients' sync at the same instant — before any record section runs.
|
|
145
|
+
**Localizing the crash from `Parameters`:** `NETSUITE_EXECUTION_MODE_*` is `<seconds>-IDLE` vs
|
|
146
|
+
`<seconds>-RUNNING`; `startModeIteration()` writes RUNNING at the *start* of a section. If a
|
|
147
|
+
frozen client's mode is stuck **IDLE** (not RUNNING), the script is dying **before** any section
|
|
148
|
+
— i.e. in the LOOKUPS/setup phase (autoload/parse error, or a lookup 403/exception). Stuck
|
|
149
|
+
**RUNNING** means it crashed *inside* that section. And because `SALES_ORDERS` advances its
|
|
150
|
+
watermark to ~now on every completed run (no 24h cap), a SALES_ORDERS watermark frozen for days
|
|
151
|
+
means no run has *completed*, even if the cron is firing. To confirm tier-wide vs per-client:
|
|
152
|
+
compare several clients' watermarks — **all frozen at the same timestamp = shared/upstream cause**
|
|
153
|
+
(e.g. this parse error), not per-client data. When the fix deploys, MODE flips to RUNNING and the
|
|
154
|
+
watermark jumps to ~now on completion (a positive live signal); the sync self-recovers and
|
|
155
|
+
backfills the gap.
|
|
156
|
+
- **Monitoring blind spot — not every client is in the NetSuite sync monitor.**
|
|
157
|
+
`App_SystemMonitor_NetSuiteIntegration` (`library/app/systemmonitor/netsuiteintegration.php`)
|
|
158
|
+
alerts ClickUp when any client's checkpoint is >48h stale, but its `CLIENT_CONFIGURATIONS` list
|
|
159
|
+
(~line 16) **omits Quad, AIG, and Growrk**. Their stalls raise no alert (Quad's 3-day July-2026
|
|
160
|
+
outage went unflagged for that reason). Follow-up: add Quad/AIG/Growrk to `CLIENT_CONFIGURATIONS`.
|
|
134
161
|
- **Seeded-but-stale = stalled sync.** Verify health by reading `LAST_SYNC_DATETIME_*` in each
|
|
135
162
|
`Client_<Id>.Parameters`: a healthy client advances to ~now every 5 min. As of 2026-06-17,
|
|
136
163
|
Quad was seeded but frozen at 2025-04/05 (not advancing) while the other 12 deployed clients
|
|
@@ -151,6 +178,19 @@ Parameters are stored **per client DB** but accessed **through the TOGa2 API**,
|
|
|
151
178
|
|
|
152
179
|
## Change history
|
|
153
180
|
|
|
181
|
+
- 2026-07-09 — **Tier-wide outage (2026-07-06 → 07-09): a dropped brace in shared `rest.php`
|
|
182
|
+
killed every client's sync for 3 days.** A bad merge (`c3caa11a`, "Merge branch '_production'
|
|
183
|
+
into TRUE-79078", 2026-06-24) silently dropped two tokens at the `listLocations()` /
|
|
184
|
+
`listPurchaseOrders()` boundary in `library/app/api/netsuite/rest.php` (~line 1119) during
|
|
185
|
+
conflict resolution — the closing `}` of `listLocations()` and the `/**` opening the
|
|
186
|
+
`listPurchaseOrders()` docblock — leaving `return $out;` immediately followed by a bare
|
|
187
|
+
` * Returns purchase orders...`, a fatal `syntax error, unexpected '*'`. `App_Api_Netsuite_Rest`
|
|
188
|
+
then failed to load entirely, so every client's autoload in the LOOKUPS phase crashed. Shipped to
|
|
189
|
+
prod via automated merge `c85e39e5` at 2026-07-06 16:17 UTC — the exact minute all clients' last
|
|
190
|
+
successful sync ran. Fix: restored the missing `}` and `/**`; verified `php -l` clean. Sync
|
|
191
|
+
self-recovered and backfilled the July 6–9 gap (Quad invoice 313319 / order 281251 / $2,356.22
|
|
192
|
+
imported, all watermarks current). See the new tier-wide-parse-error and monitor-blind-spot
|
|
193
|
+
gotchas above. (bala)
|
|
154
194
|
- 2026-06-30 — The IF section's `syncItemFulfillmentFromNetsuite` (`library/app/api/toga2.php`) now
|
|
155
195
|
imports **all** fulfillment statuses (`['_picked','_packed','_shipped']`, not just shipped),
|
|
156
196
|
stamping/advancing the IF stage from `c_netsuiteInternalItemFulfillmentStatus`. Full mechanics
|
|
@@ -5,7 +5,7 @@ project: Worker
|
|
|
5
5
|
client: tow-foundation
|
|
6
6
|
type: client-feature
|
|
7
7
|
status: active
|
|
8
|
-
updated: 2026-
|
|
8
|
+
updated: 2026-07-09
|
|
9
9
|
owners: ["rgirish"]
|
|
10
10
|
files:
|
|
11
11
|
- worker2/Worker/Client/TowFoundation.php
|
|
@@ -57,14 +57,15 @@ Credit Card Receipts/
|
|
|
57
57
|
5. **Pass 1 — Extract** — for each receipt:
|
|
58
58
|
- Enforce type + size limit: unsupported mime types (`application/octet-stream`) throw immediately; 4 MB cap for images, 10 MB for documents
|
|
59
59
|
- Download file bytes from SharePoint
|
|
60
|
-
- **`.docx` files bypass Talos** and route to `extractDocxData()` (Talos returns HTTP 500 on a `.docx` MIME type — it only accepts PDF/image). That method opens the docx as a ZIP, extracts `word/document.xml`, strips tags, and regex-parses `
|
|
60
|
+
- **`.docx` files bypass Talos** and route to `extractDocxData()` (Talos returns HTTP 500 on a `.docx` MIME type — it only accepts PDF/image). That method opens the docx as a ZIP, extracts `word/document.xml`, strips tags, and regex-parses ride lines into `line_items[]`. The ride-line regex (in both `extractDocxData()` and `parseDocxRideLines()`) accepts **slash or dot** date separators (`[\/.]` — e.g. `05/06/2025` and `05.06.2025`) and treats the `[:\-]?` description delimiter as **optional**. Dash-separated dates (`05-06-2025`) are intentionally **not** supported — a dash would clash with the `:`/`-` description delimiter. A docx that matches zero ride lines throws `ReceiptProcessingException` and produces no Excel, so a new date format silently zero-outputs — widen the regex when a cardholder introduces one
|
|
61
61
|
- All other types POST to Talos AI `/api/ai/generate` → structured `{vendor_name, invoice_date, total, payment_memo, category, ...}`
|
|
62
62
|
- **Year guard on AI dates** — if Talos returns an `invoice_date` whose year is more than 1 year from the current year (AI hallucination on two-digit year inputs, e.g. `5/13/76` → 1976, `5/28/28` → 2028), the year is clamped to the current year while month/day are preserved
|
|
63
|
-
-
|
|
63
|
+
- **Payment memo — the statement `Notes` column is authoritative.** `loadStatementExcel()` detects a `Notes` column (exact-match `"notes"`/`"note"` first; a substring fallback runs only if no exact header was found, guarded by `$notesExact`) and attaches a `notes` field to each parsed row. In `Run()`, if the matching statement row has a **non-empty `notes` value**, it is used **verbatim** as `payment_memo` — this is the cardholder's own hand-written memo (e.g. `"Postage: Board materials shipped to Frank Tow"`) and beats any AI-inferred memo. Only if the row has no notes does processing fall back to the prior AI/description behavior. This **supersedes** the earlier `isActivityHeadingMemo()` allow-list, which dropped the cardholder's Notes entirely (it only detected columns containing `"description"`/`"memo"`) and rejected valid headings like `Communications:`, `Postage:`, `Office Supplies:`, `RF:` — shipping AI memos in their place.
|
|
64
|
+
- **Statement row matching.** `matchStatementCandidate()` returns the whole matched statement row (± $0.01 amount, then date tie-break). `matchStatementRow()` (description fallback, best-effort) and `matchStatementNotes()` (returns the row's `notes`) both delegate to it. `matchStatementCandidate($strict)`: in **strict** mode, when multiple rows share an amount and none resolves by date, it returns `null` and logs — rather than guessing an arbitrary first candidate. `matchStatementNotes()` uses `strict=true` (attaching the wrong cardholder's memo is worse than none); the description fallback stays non-strict/best-effort.
|
|
64
65
|
- **Zero-total fallback** — if the extracted/docx total is 0, `lookupStatementTotalByVendor()` derives the amount from the billing statement (see *Statement structure* below)
|
|
65
66
|
- Build base filename (no suffix yet) and store in `$pendingRenames`
|
|
66
67
|
- On extract failure → move to `Archive/exception/` immediately
|
|
67
|
-
6. **Deduplication** — after Pass 1, receipts
|
|
68
|
+
6. **Deduplication** — after Pass 1, receipts are deduplicated on the **SharePoint drive-item `fileId` only** (a true re-upload is the same drive item; distinct files are distinct items). Duplicates are logged as `DUPLICATE skipped` and dropped. **Do not** key dedup on `(personName, invoice_date, vendor_name, total)` — that collided genuinely distinct same-day/same-amount charges (e.g. two $26 Metro-North legs, two $3 subway swipes) and silently dropped one, so 2 of Michael's 5 charges never reached the sheet. Distinct files that produce the same base filename are handled downstream by the `_a`/`_b` suffix logic.
|
|
68
69
|
7. **Collision detection** — count base name occurrences across remaining pending renames
|
|
69
70
|
8. **Pass 2 — Move** — for each pending rename:
|
|
70
71
|
- If base name appears more than once, assign `_a`, `_b`, `_c`... suffix to **all** colliding files (including the first)
|
|
@@ -72,7 +73,19 @@ Credit Card Receipts/
|
|
|
72
73
|
- **Multi-entry docx expansion** — a `.docx` whose `extractDocxData()` produced multiple `line_items[]` (e.g. a subway-rides document with 14 dated trip entries) is expanded into one Excel row per entry
|
|
73
74
|
8. **Excel generation** — one `.xlsx` per person (PhpSpreadsheet); columns: Row #, Account Name, QB Vendor, Payment Amount, Date, Payment Method, Payment Memo, QB Description, Class, Category, Payment Account, Ref No.
|
|
74
75
|
9. **SharePoint upload** — each Excel uploaded to root-level `Credit Card Receipts/3. QB Excel/` via Graph API PUT (shared folder for all persons)
|
|
75
|
-
10. **Summary email** — sent with Excel files attached; To: Jheanelle, CC: Magdalena, BCC: devteam@togatech.com
|
|
76
|
+
10. **Summary email** — sent with Excel files attached; To: Jheanelle, CC: Magdalena, BCC: devteam@togatech.com. `sendSummaryEmail()` takes a `$warnings` param and renders an **"Amounts to verify"** section listing rows whose amount could not be reconciled against the statement (see below).
|
|
77
|
+
|
|
78
|
+
### Amount reconciliation (flag-only)
|
|
79
|
+
A non-docx receipt's `payment_amount` comes **solely from AI OCR** and is **never** overridden by
|
|
80
|
+
the statement `Amount` — the statement is consulted only for the memo. This is systemic for every
|
|
81
|
+
JPG/PDF receipt (e.g. a Kellari Taverna charge OCR'd a handwritten total as `$548.11` vs. the
|
|
82
|
+
statement's `$538.11`). To surface these without changing the number, `statementHasAmount()` checks
|
|
83
|
+
whether the AI total matches any statement charge within a cent. In `Run()`, for **non-docx** receipts
|
|
84
|
+
with `total > 0` **when a statement is loaded for that cycle**, if no statement amount matches, a
|
|
85
|
+
non-fatal entry is added to `$warnings` (**the row is still emitted**) and rendered in the summary
|
|
86
|
+
email's "Amounts to verify" section for human review before QuickBooks import. Docx receipts (amount
|
|
87
|
+
sourced from the statement) and receipts with no loaded statement are excluded. Auto-overriding
|
|
88
|
+
`payment_amount` from the statement was **deliberately deferred** as higher-risk.
|
|
76
89
|
|
|
77
90
|
### Renamed file format
|
|
78
91
|
```
|
|
@@ -87,6 +100,18 @@ Amex billing cycle runs 3rd-to-3rd. `computeRefNo()` returns the cycle-end date
|
|
|
87
100
|
- Charge on or before the 3rd → cycle ends on the 3rd of the same month
|
|
88
101
|
- Charge after the 3rd → cycle ends on the 3rd of the next month
|
|
89
102
|
|
|
103
|
+
### Excel filename — derived from the billing-cycle end, not the charge date
|
|
104
|
+
`buildExcelFileName(..., $billingCycleFolder)` uses the **cycle end**, not the first row's
|
|
105
|
+
charge date, to pick the reporting month — so April charges that belong to the
|
|
106
|
+
`"Amex ending in 05-03-2026"` cycle produce a `_05_2026` file that agrees with the per-row
|
|
107
|
+
Ref No. (which is also cycle-end-based). `parseCycleEndDate()` parses `MM-YYYY` from the
|
|
108
|
+
cycle folder name (anchored to `"ending in"`, with a bare `MM-DD-YYYY` fallback). If the
|
|
109
|
+
cycle can't be resolved, it falls back to the charge date. `resolvePersonCycle()` picks a
|
|
110
|
+
person's cycle from `$personCycles[person][cycleFolder]` (collected in Pass 2); if a run
|
|
111
|
+
spans **multiple** cycles for one person it warns and uses the first (see open items). The
|
|
112
|
+
resolved cycle is passed to all three `buildExcelFileName()` call sites (upload + both
|
|
113
|
+
`sendSummaryEmail()` sites) so the uploaded filename and the email attachment always match.
|
|
114
|
+
|
|
90
115
|
### Billing statement structure (Amex xlsx)
|
|
91
116
|
|
|
92
117
|
The Amex statement Emily submits is a **multi-sheet** xlsx — one sheet per weekly billing
|
|
@@ -207,9 +232,11 @@ Fatal errors send only to `NOTIFY_EMAIL_DEV` (no CC/BCC).
|
|
|
207
232
|
names with special characters (spaces, dots). Always resolve to ID first.
|
|
208
233
|
- **`ensureFolderPath()` for "3. QB Excel"** — the folder name contains a dot and space;
|
|
209
234
|
`ensureFolderPath` handles this correctly via `rawurlencode` on each path segment.
|
|
210
|
-
- **Statement matching tolerance** —
|
|
211
|
-
|
|
212
|
-
|
|
235
|
+
- **Statement matching tolerance** — matching uses `abs($row['amount'] - $amount) < 0.01`, then a
|
|
236
|
+
date tie-break for multiple same-amount rows. When even the date tie-break is ambiguous, behavior
|
|
237
|
+
depends on the caller's `$strict` flag: **strict** (`matchStatementNotes()`) returns `null` and
|
|
238
|
+
logs rather than guessing (a wrong authoritative memo is worse than none); **non-strict**
|
|
239
|
+
(`matchStatementRow()` description fallback) returns the first candidate, best-effort.
|
|
213
240
|
- **No client DB** — there is no audit trail in MySQL. All state lives in SharePoint folder
|
|
214
241
|
structure and email. If a run is interrupted mid-way, some receipts may be archived
|
|
215
242
|
without a corresponding Excel row.
|
|
@@ -228,14 +255,29 @@ Fatal errors send only to `NOTIFY_EMAIL_DEV` (no CC/BCC).
|
|
|
228
255
|
- **Archive paths must use the actual SharePoint folder name, not the normalized name** — the normalized person name (` CC receipts` stripped, spaces replaced) is for **map lookups only**. When building the SharePoint archive path, use the *actual* folder name (e.g. `"Emily Tow CC receipts"`), not the normalized `"Emily Tow"` — otherwise the PATCH move 404s because the path segment does not exist. Keep the normalized name and the real folder name as separate values.
|
|
229
256
|
- **`billingCycle` filter is suffix-match, not exact** — always pass just the date portion (`"06-03-2026"`), not the full folder name. Passing the full name (`"Amex ending in 06-03-2026"`) also works but would miss Mastercard folders.
|
|
230
257
|
- **Vendors not in QB vendor list** — as of June 2026 these vendors are not in Tow Foundation's QuickBooks, so `qb_vendor` is blank for their charges until the client adds them to QB and the vendor list is refreshed: **Ole Mole**, **AMORE PIZZA CAFE**, **Green & Tonic (New Canaan)**, **NCFP (National Center for Philanthropy)**, **Rippling**, **Langan's**. This is a client action, not a code fix.
|
|
231
|
-
- **Duplicate detection is within-run only** —
|
|
258
|
+
- **Duplicate detection is within-run and `fileId`-keyed only** — dedup keys on the SharePoint
|
|
259
|
+
drive-item `fileId`, so it catches only a literal re-upload of the same item present in the
|
|
260
|
+
current run. It deliberately does **not** dedup on person/date/vendor/amount (that dropped
|
|
261
|
+
distinct same-amount charges); genuinely distinct files with the same content are all kept.
|
|
232
262
|
- **Unmapped AI categories pass through** — if the AI returns a category string not in `TowFoundationCategories.php`, `mapCategory()` logs a warning and writes the raw AI string to the Excel. Check error logs after a run if the Category column looks odd; add the new value to the map file to fix it.
|
|
233
263
|
- **Short vendor acronyms (< 4 chars) require whole-word match** — the substring pass uses a word-boundary regex for needles under 4 chars to avoid false positives (e.g. `"UPS"` inside `"USPS"`). If a short vendor name is not matching, check that the QB vendor list entry starts or ends with the acronym as a whole word.
|
|
234
264
|
- **Restoring archived receipts** — use the committed `MoveBack` action (see *MoveBack action* above), not a one-shot CLI script. It moves all non-Excel receipt files from a person's `Archive/` back into the named billing-cycle folder. (The earlier throwaway `/tmp/tow_restore_archives.php` script is obsolete.)
|
|
235
265
|
- **Mixed-cycle receipts can collect in Archive** — receipts from a prior run can linger in a person's `Archive/` under a different billing cycle (e.g. January receipts found alongside May during an Emily Tow restore). `MoveBack` moves **everything** in `Archive/` back, so after a restore, verify the receipts belong to the cycle you intend to reprocess; stray off-cycle files should be moved (via Graph API) to their own billing-cycle folder so they are not reprocessed with the wrong cycle.
|
|
236
266
|
|
|
267
|
+
## Open items / known-deferred
|
|
268
|
+
|
|
269
|
+
- **Multi-cycle runs merge to one file** — if a single run spans multiple billing cycles for one
|
|
270
|
+
person, all rows still merge into one Excel (named after the first cycle) with a logged warning.
|
|
271
|
+
Full per-`(person, cycle)` output split is deferred.
|
|
272
|
+
- **Dash-separated docx dates unsupported** — `05-06-2025` is intentionally not parsed (dash clashes
|
|
273
|
+
with the description delimiter); only slash and dot separators work.
|
|
274
|
+
- **Amount reconciliation is flag-only** — mismatched OCR amounts are surfaced in the "Amounts to
|
|
275
|
+
verify" email section but never auto-corrected from the statement; auto-override was deferred as
|
|
276
|
+
higher-risk.
|
|
277
|
+
|
|
237
278
|
## Change history
|
|
238
279
|
|
|
280
|
+
- 2026-07-09 — **Client QA pass on a production run — six fixes (code-only, not deployed).** (1) Statement `Notes` column is now the **authoritative** payment memo, used verbatim; `loadStatementExcel()` detects `Notes` (exact-match first, guarded substring fallback) — replaces the `isActivityHeadingMemo()` allow-list that dropped hand-written memos (fixed Angela, Emily, Katy, Meghan, Nadia). (2) `matchStatementCandidate($strict)` returns `null` on ambiguous same-amount matches for the Notes memo instead of guessing; `matchStatementNotes()` is strict, description fallback stays best-effort. (3) Dedup now keys on SharePoint `fileId` only — the old person|date|vendor|amount key dropped distinct same-amount charges (Michael lost 2 of 5). (4) docx ride-line regex now accepts dot dates (`[\/.]`) with optional `[:\-]?` delimiter — Jheanelle's dot-format subway docx previously matched 0 lines and produced no output. (5) `buildExcelFileName()` now derives the reporting month from the billing-cycle end (`parseCycleEndDate()`/`resolvePersonCycle()`) not the charge date, so the filename agrees with the Ref No. (Diane). (6) Systemic discovery: non-docx `payment_amount` is AI-OCR only, never reconciled against the statement `Amount` (Meghan $548.11 vs $538.11); added flag-only `statementHasAmount()` + `$warnings` "Amounts to verify" email section — auto-override deferred. (rgirish)
|
|
239
281
|
- 2026-06-29 — **Reversed the over-aggressive memo override.** Replaced `isBareTranportationMemo()` (added earlier same day) with `isActivityHeadingMemo()`: the statement memo now overrides the AI memo **only** when it is a proper activity-purpose heading (e.g. `Grantee meeting:`, `Peer meeting:`), not whenever the AI memo started with `"Transportation:"`. The previous approach replaced semi-useful AI memos with raw bank strings (`"AplPay LYFT …"`, `"UBER"`) on ride-share/taxi rows where the statement has no activity heading — degraded 8 rows on Emily Tow's statement (1, 17, 18, 21, 26, 31, 32, 34). New priority: statement heading > AI memo, with AI memo kept for any non-heading statement description. (rgirish)
|
|
240
282
|
- 2026-06-29 — `payment_memo` override now also fires on a **bare** `"Transportation:"` AI memo, not just an empty one: new `isBareTranportationMemo()` lets the cardholder's statement Excel (the authoritative source for activity-purpose headings like "Peer meeting:", "Grantee event:") override the AI's generic transportation heading. Fixes wrong headings on Emily Tow May 2026 transit rows. Added the committed `MoveBack` action (+ `collectFilesRecursiveRaw()`) to restore a person's archived receipts into a billing-cycle folder for reprocessing, replacing the throwaway `/tmp/tow_restore_archives.php` script. (rgirish)
|
|
241
283
|
- 2026-06-25 — Statement parsing hardened for Emily Tow's May 2026 Amex xlsx: `loadStatementExcel()` now iterates `getAllSheets()` (4 weekly-period sheets) instead of `getActiveSheet()` and captures the `Receipt` column; `lookupStatementTotalByVendor()` now **sums** all receipt-label-matched rows (14 × $3.00 "Subway Rides" = $42.00) before falling back to description-keyword match — fixes subway totals reading $0/$9.00. Added `extractDocxData()` so `.docx` receipts bypass Talos (Talos 500s on docx), parsing `MM/DD/YY: description` line items into per-entry Excel rows. Added a year guard clamping AI-hallucinated `invoice_date` years (e.g. 1976, 2028) to the current year. Fixed archive-path bug: SharePoint move now uses the actual folder name (`"Emily Tow CC receipts"`) not the normalized name, fixing 404 on archive moves. 5 more vendors found missing from the QB vendor list (client action). (rgirish)
|
package/package.json
CHANGED