toga-ai 1.0.326 → 1.0.327

package/knowledge/clients/tow-foundation/features/receipt-processing.md

@@ -29,6 +29,8 @@ Optional filter parameters: `limit` (int — caps count processed, **not a dry-r
 
 The monthly production procedure is **two explicit per-card runs** (Amex + Ryan Farrell's Mastercard), never a single month sweep — see *Monthly production run procedure* below.
 
+**Before a real run, use `Preview` to see who actually has receipts for a cycle** (read-only dry-run — downloads nothing, moves nothing, sends no email). See *Preview action (read-only dry-run)* below.
+
 ## Key files / entry points
 
 - `TowFoundation.php` — base class; holds all email constants and `initialize()` (empty — no client DB)
@@ -195,6 +197,36 @@ the valid category names; keep them in sync with the map keys.
 to their QuickBooks Class and Payment Account strings. Ryan Farrell uses Bank of America
 Mastercard; all others use AMEX Open Credit Card.
 
+### Preview action (read-only dry-run)
+`Preview(?string $year = null, ?string $person = null, ?string $billingCycle = null): string` is
+the read-only counterpart to `Run()`. It reuses `walkReceiptsFolder()` and the **same**
+`year`/`person`/`billingCycle` filters, but **only tallies** — it downloads nothing, extracts
+nothing, moves nothing, and sends no email. This is the supported way to answer "who actually has
+receipts for this cycle / is this month mostly empty?" **before** a real, side-effecting `Run()`
+(there is otherwise no dry-run — see *Production-safety facts*).
+
+```json
+{
+  "action": "Client/TowFoundation/ProcessReceipts/Preview",
+  "parameters": { "year": "2026", "billingCycle": "07-03-2026" }
+}
+```
+
+Returns pretty JSON:
+```
+{ filters, totalReceipts, personsWithReceipts,
+  persons: [ { person, receipts (count), cycles: [folder...], fileTypes: {ext => count},
+               sampleFiles: [up to 3 names], hasStatement (bool) } ],
+  personsWithNoReceipts: [...] }
+```
+
+`sampleFiles` + `fileTypes` were added so a human can confirm the counted files are genuine
+receipts (`pdf`/`jpg`/`png`/`docx`) rather than statements. **Read `persons[]` as the truth:** a
+cardholder is empty for the cycle iff they are **absent from `persons[]`**. `personsWithNoReceipts`
+is a softer signal — it is derived from people who have a **statement** file but no receipts this
+cycle, and it also surfaces stray top-level folders (e.g. `"Processed"`) as pseudo-"persons"
+(cosmetic noise). Do not treat presence in `personsWithNoReceipts` as authoritative.
+
 ### MoveBack action (restore archived receipts for reprocessing)
 `MoveBack(string $person, string $billingCycle, ?string $year = null): string` is a public
 static action that walks the `Archive/` folder under a person's year directory and moves all
@@ -259,7 +291,9 @@ Ryan Farrell's Mastercard cycle:
   list comes entirely from the `NOTIFY_*` constants in `Worker/Client/TowFoundation.php` (see
   *Email routing*) — the request cannot change who is emailed. Secret/config values (SharePoint
   credentials, drive/site IDs) live in `TowFoundation.php` constants and `_Config`, never in the
-  request or this doc.
+  request or this doc. **SharePoint credentials** specifically live in the
+  `[sharepoint_towfoundation]` section of the per-environment Config ini (e.g. `dev-rohan-mac.ini`)
+  and are read via `_Config::sharepoint_towfoundation()` — document only *where*, never the values.
 
 ## Email routing
 
@@ -318,7 +352,8 @@ Fatal errors send only to `NOTIFY_EMAIL_DEV` (no CC/BCC).
   recorded, a fallback `error_log()`s and defaults to `{personName}/{currentYear}`. (Before
   2026-07-13 all persons' Excels went to one shared root `Credit Card Receipts/3. QB Excel/`.)
 
-- **Person folder name normalization** — SharePoint folders are named `"Brent Peterkin CC receipts"` but `CLASS_MAP` / `PAYMENT_ACCOUNT_MAP` keys are just `"Brent Peterkin"`. The ` CC receipts` suffix is stripped via regex in `walkReceiptsFolder()`. Without this, Class and Payment Account columns are blank for those persons.
+- **Person folder name normalization — map keys MUST equal the folder-derived name** — SharePoint folders are named `"Brent Peterkin CC receipts"` but `CLASS_MAP` / `PAYMENT_ACCOUNT_MAP` keys are the **normalized folder name** (folder name minus the ` CC receipts` suffix, stripped via regex in `walkReceiptsFolder()`). The person filter and both map lookups all key on this same folder-derived name, so **a map key that does not exactly equal `folder − " CC receipts"` is never hit**. Both maps fall back to `''` on a miss **with no warning**, so a mismatched cardholder silently ships **blank Class + Payment Account** columns. Confirmed real: the folder `"Ligia Marroquin Soto CC receipts"` normalizes to `"Ligia Marroquin Soto"`, but both maps were keyed `"Ligia Marroquin"` (no "Soto") — her rows blanked out silently until the keys were renamed to `"Ligia Marroquin Soto"` (`CLASS_MAP` ⇒ `Administration:Operations`, `PAYMENT_ACCOUNT_MAP` ⇒ `AMEX Open Credit Card:Ligia Marroquin-Soto Amex CC`). When adding a cardholder, copy the exact normalized folder name as the key.
+- **Unmapped cardholder → silent blank columns** — a cardholder with a receipts folder but **no** entry in `CLASS_MAP`/`PAYMENT_ACCOUNT_MAP` gets blank Class + Payment Account (same silent `''` fallback as above). As of 2026-07-13, **Michael Zuber Zander** has a receipts folder but is in **neither** map (and currently has zero receipts in any cycle), so he is left unmapped/uncommented pending his QB Class + Payment Account values from the client. Before running a cardholder for the first time, confirm they exist in both maps.
 - **Archive paths must use the actual SharePoint folder name, not the normalized name** — the normalized person name (` CC receipts` stripped, spaces replaced) is for **map lookups only**. When building the SharePoint archive path, use the *actual* folder name (e.g. `"Emily Tow CC receipts"`), not the normalized `"Emily Tow"` — otherwise the PATCH move 404s because the path segment does not exist. Keep the normalized name and the real folder name as separate values.
 - **`billingCycle` filter is suffix-match, not exact** — always pass just the date portion (`"06-03-2026"`), not the full folder name. Passing the full name (`"Amex ending in 06-03-2026"`) also works but would miss Mastercard folders.
 - **Missing per-cycle statement → no Notes memos (data dependency, not a bug)** — statements
@@ -371,9 +406,21 @@ Fatal errors send only to `NOTIFY_EMAIL_DEV` (no CC/BCC).
 - **Subway docx line-item memos come from the docx, by design** — the per-ride memos on a subway
   `.docx` are sourced from the docx line-item description, not the statement `Notes` column. This is
   intentional; do not "fix" it by routing them through `matchStatementNotes()`.
+- **July 2026 cycle reality (`Preview`, as of 2026-07-13)** — Amex `"07-03-2026"`: **72** real
+  receipts across **9** people (Nadia Alia 43, Emily Tow 11, Meghan Lowney 8, Johany Bedon 3, Angela
+  Leis 2, Magdalena Minta 2, Eileen Wiseman 1, Kathryn Lockhart 1, Ligia Marroquin Soto 1). Empty for
+  July (only earlier cycles): Diane Sierpina, Brent Peterkin, Susan Ransden. **Two cardholders with
+  July receipts but NO statement file** → their memos will be AI-inferred (see the missing-statement
+  gotcha above): **Ryan Farrell** (Mastercard, expected) and **Magdalena Minta**. Confirms per-person
+  runs are correct: people span multiple cycles and the `billingCycle` filter narrows to the target
+  cycle.
+- **Michael Zuber Zander unmapped** — see the *Unmapped cardholder* gotcha above; he is in neither
+  map and has zero receipts currently. Resolve his QB Class + Payment Account with the client before
+  he first appears in a cycle.
 
 ## Change history
 
+- 2026-07-13 — **Read-only Preview action + Ligia map-key fix + July cycle findings.** (1) BUILT: `Preview(?year, ?person, ?billingCycle): string` — a true dry-run that reuses `walkReceiptsFolder()` and the same filters as `Run()` but only tallies (downloads/extracts/moves nothing, sends no email), returning pretty JSON with per-person receipt counts, cycles, fileTypes, up-to-3 sampleFiles, and hasStatement, plus a softer `personsWithNoReceipts` list. Read `persons[]` as the authoritative empty signal; `personsWithNoReceipts` is derived from statement-only folders and also surfaces stray top-level folders (e.g. "Processed") as cosmetic noise. (2) FIXED: `CLASS_MAP`/`PAYMENT_ACCOUNT_MAP` keys for Ligia Marroquin Soto were `"Ligia Marroquin"` (no "Soto") — never hit against the folder-derived `"Ligia Marroquin Soto"`, so her Class + Payment Account silently blanked (both maps fall back to `''` with no warning). Renamed keys to `"Ligia Marroquin Soto"` (Class ⇒ `Administration:Operations`, Payment Account ⇒ `AMEX Open Credit Card:Ligia Marroquin-Soto Amex CC`); broadened the person-folder-normalization gotcha — map keys MUST equal `folder − " CC receipts"`. (3) DISCOVERED: July Amex `"07-03-2026"` has 72 receipts across 9 people (Nadia Alia 43 …); Diane Sierpina, Brent Peterkin, Susan Ransden empty for July; Ryan Farrell + Magdalena Minta have July receipts but no statement → AI-inferred memos; **Michael Zuber Zander** has a receipts folder but is in neither map (and zero receipts) → left unmapped pending client-provided QB Class + Payment Account. Also recorded that SharePoint creds live in the `[sharepoint_towfoundation]` Config ini section, read via `_Config::sharepoint_towfoundation()`. (rgirish)
 - 2026-07-13 — **Per-person QB Excel location fix + two production findings.** (1) FIXED: `uploadExcelToSharePoint()` now writes each person's generated QB Excel to that person's own `Credit Card Receipts/{Person} CC receipts/{Year}/3. QB Excel/` subfolder (matching the docblock and client expectation) instead of a single shared root `3. QB Excel/`; signature is now `(accessToken, driveId, personFolderName, year, excelName, tmpFile)`, fed by a `$personFolders` map populated first-write-wins in Pass 1 with an `error_log` fallback to `{personName}/{currentYear}`. (2) DISCOVERED (production-critical gotcha): the statement-Notes lookup is EXACT string equality between the statement filename (`cycleKey`) and the receipt's billing-cycle FOLDER name — no date/fuzzy fallback — so a card whose folder has no identically-named statement silently skips the authoritative Notes memo and falls back to AI inference with zero signal (confirmed for Ryan Farrell's Mastercard folder vs. an Amex-named statement); also noted `computeRefNo()` hard-codes cycle-end day `03`, giving Mastercard rows a `…03…` Ref No. (pre-existing, unchanged). (3) DECIDED: monthly production procedure is two explicit per-card `Run` invocations (Amex + Ryan Farrell's Mastercard), NOT a "MM-YYYY month sweep" — the sweep prototype was deliberately reverted because `PAYMENT_ACCOUNT_MAP` is keyed per-person not per-card and would misattribute the QB Payment Account for anyone holding two cards in a swept month; recorded production-safety facts (no dry-run — `limit` still archives + emails; file moves reversible via `MoveBack` but email is not recallable; recipients are compile-time `NOTIFY_*` constants). (rgirish)
 - 2026-07-09 — **Local reprocess-all-cardholders verification pass — three more fixes + two data/ops findings (code-only, uncommitted).** These fixes were found *after* the six-fix pass earlier the same day, while re-running every cardholder locally and inspecting the uploaded QB files. (1) **Statements-folder detection matched plural `reports` only** — `walkReceiptsFolder()` now matches `stripos(name,'report')` (singular), so `"{Name} report"` folders (Angela Leis, Kathryn Lockhart, Meghan Lowney, Eileen Wiseman) actually load their statement; without this the earlier Notes-memo fix silently did nothing for them in a real `Run()` (folder-name matching is the gate — verifying `matchStatementNotes()` in isolation is not enough). (2) **docx run-boundary word-splitting** — `</w:r>` was replaced with a space, injecting spaces mid-word (`"Innoc ence"`, `"Armstron g"`, `"202 6 . 0 6 .0 3"`) in Emily Tow's subway memos; now replaced with `''` at both sites (`extractDocxData`, `parseDocxRideLines`) — only `</w:p>` is a real line break. (3) **docx "Closing Date" header mis-parsed as first ride** — the ride regex read `"AMEX Closing Date 2026.06.03"` as a ride and swallowed the real first ride; now stripped upfront via `preg_replace('/(?:AMEX|Visa)\s+Closing\s+Date\s+\d{4}[.\/-]\d{1,2}[.\/-]\d{1,2}/i','',$text)` (kept 4-digit dot dates for Jheanelle rather than narrowing ride dates). Data/ops findings (no code change): **Nadia Alia's** June statement is absent from SharePoint → all her memos AI-inferred until the client uploads `"Amex ending in 06-03-2026.xlsx"` with a Notes column; local worker HTTP endpoint 500s on direct `{action,parameters}` calls (post 2026-07-07 `Core.WorkerJobs` dependency absent in dev) → reprocess a cycle via CLI + `MoveBack` (debug_mode redirects mail). Verified correct: Angela, Emily (incl. 14 subway line items), Kathryn, Meghan (Kellari $548.11 flagged), Michael (all 5 charges). Nadia pending client statement; Jheanelle + Diane not yet reprocessed. (rgirish)
 - 2026-07-09 — **Client QA pass on a production run — six fixes (code-only, not deployed).** (1) Statement `Notes` column is now the **authoritative** payment memo, used verbatim; `loadStatementExcel()` detects `Notes` (exact-match first, guarded substring fallback) — replaces the `isActivityHeadingMemo()` allow-list that dropped hand-written memos (fixed Angela, Emily, Katy, Meghan, Nadia). (2) `matchStatementCandidate($strict)` returns `null` on ambiguous same-amount matches for the Notes memo instead of guessing; `matchStatementNotes()` is strict, description fallback stays best-effort. (3) Dedup now keys on SharePoint `fileId` only — the old person|date|vendor|amount key dropped distinct same-amount charges (Michael lost 2 of 5). (4) docx ride-line regex now accepts dot dates (`[\/.]`) with optional `[:\-]?` delimiter — Jheanelle's dot-format subway docx previously matched 0 lines and produced no output. (5) `buildExcelFileName()` now derives the reporting month from the billing-cycle end (`parseCycleEndDate()`/`resolvePersonCycle()`) not the charge date, so the filename agrees with the Ref No. (Diane). (6) Systemic discovery: non-docx `payment_amount` is AI-OCR only, never reconciled against the statement `Amount` (Meghan $548.11 vs $538.11); added flag-only `statementHasAmount()` + `$warnings` "Amounts to verify" email section — auto-override deferred. (rgirish)

package/package.json

@@ -1,6 +1,6 @@
 {
   "name": "toga-ai",
-  "version": "1.0.326",
+  "version": "1.0.327",
   "description": "TOGA Technology Team Claude Knowledge System — shared AI coding harness with skills, knowledge base CLI, and project installer for Claude Code.",
   "keywords": [
     "claude",