toga-ai 1.0.987 → 1.0.989

This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
@@ -7,6 +7,7 @@
7
7
  | [API Request Logging & Retention (App_ApiTransaction → Logs.API / Logs_<Tenant>)](features/api-request-logging-and-retention.md) | Where every outbound API call is logged (1.0 `Logs.API` vs the 2.0 `Logs_<Tenant>` cluster), how the routing is chosen, and the 30-day retention trim; open befo |
8
8
  | [Where a new App_ class goes — the app/ folder IS a behavioral contract](features/app-class-placement-base-contracts.md) | Where a new `App_` class goes — each `app/` folder's base class carries real inherited behavior; open before creating any class under `app/model|client|api`. |
9
9
  | [App_Sso — Reusable 1.0 SSO Initiation (SP-initiated SAML via saml.togahub.com)](features/app-sso-initiation.md) | Reusable 1.0 SP-initiated SAML SSO via `saml.togahub.com` (`App_Sso::initiate`); the canonical doc to follow for any "add SSO to `<1.0 app>`" request. |
10
+ | [Carrier detection from a tracking number (App_Misc) — patterns, check digits, shipping-method fallback](features/carrier-detection-from-tracking-number.md) | How 1.0 decides which carrier a tracking number belongs to (`App_Misc`, `library/app/misc.php`): certain vs shared-shape patterns, the FedEx check digit, and th |
10
11
  | [Cron Execution Monitoring (App_Framework check-in/out → CronJobExecutions)](features/cron-execution-monitoring.md) | 1.0 cron check-in/out lifecycle via `App_Framework::cronInitialization()`/`cronFinished()` → `Common.CronJobExecutions`; open to monitor a 1.0 cron or read its |
11
12
  | [1.0 MySQL Connection Lifecycle — wait_timeout, the 2006 Reconnect Gap, keepAlive/reconnect/queryForked](features/db-connection-lifecycle-and-reconnect.md) | How a 1.0 mysqli connection dies during a long external call ("MySQL server has gone away"), why the built-in 2006 reconnect never fired, and the three helpers |
12
13
  | [Diagnostic Dialog — View Recommended Services Routing](features/diagnostic-dialog-view-recommended-services.md) | TOGa Refresh 2026 "View Recommended Services" routing — per-device diagnostic `resultCode` decides serviceselection vs techsupport; open when touching that rout |
@@ -0,0 +1,170 @@
1
+ ---
2
+ title: Carrier detection from a tracking number (App_Misc) — patterns, check digits, shipping-method fallback
3
+ framework: "1.0"
4
+ repo: library
5
+ project: Library
6
+ client: shared
7
+ type: feature
8
+ status: active
9
+ updated: 2026-10-09
10
+ owners: [bala]
11
+ files:
12
+ - library/app/misc.php
13
+ - library/app/api/toga2.php
14
+ - library/app/api/netsuite/rest.php
15
+ - dbchanges2/Client_Nychh/2026-10-09 - NewYorkMinuteShippingCarrier.sql
16
+ related:
17
+ - ./netsuite-suiteql-rest-shim.md
18
+ - ./toga2-api-client-and-bridge.md
19
+ - ../../worker/features/netsuite-togasupply-per-client-sync.md
20
+ - ../../../../2.0/apps/_underscore/features/shipping-carriers-and-accounts.md
21
+ - ../../../../clients/nychh/features/transfer-order-notification-emails.md
22
+ - ../../../../clients/compass-usa/features/odp-strategic-systems-asn-csv-import.md
23
+ ---
24
+
25
+ How 1.0 decides which carrier a tracking number belongs to (`App_Misc`, `library/app/misc.php`): certain vs shared-shape patterns, the FedEx check digit, and the shipping-method fallback for local couriers. Open before touching a carrier pattern, a tracking link, or anything that writes `TrackingNumbers.shippingCarrierId`.
26
+
27
+ ## Summary
28
+
29
+ `App_Misc` is the **only** place 1.0 derives a carrier from a tracking number. Entry point
30
+ `findShippingCarrierForTrackingNumber(string $trackingNumber, string $shippingMethodCarrierName = '')`
31
+ (`misc.php:601`), with `getCarrierFromTrackingNumber()` (`:654`) as the name-returning wrapper that
32
+ ~20 worker crons use for email tracking links.
33
+
34
+ **The tracking number decides first. The shipping method's carrier is a fallback, never an override.**
35
+ Before 2026-10-09 the NetSuite item-fulfillment sync wrote tracking numbers with **no**
36
+ `shippingCarrierId` at all, so emails and screens showed no carrier and no tracking link.
37
+ `App_Api_Toga2::syncItemFulfillmentFromNetsuite` (`library/app/api/toga2.php` ~8355-8410) now reads the
38
+ fulfillment's shipping-method carrier **once per fulfillment** and passes it in.
39
+
40
+ ## How it works
41
+
42
+ Decision order inside `findShippingCarrierForTrackingNumber()`:
43
+
44
+ 1. `stripPunctuationAroundTrackingNumber()` (`:755`) — trims **edges only**.
45
+ 2. `listCertainCarrierPatterns()` (`:678`) — shapes **one** carrier owns. Trusted over whatever the
46
+ shipping system recorded.
47
+ 3. FedEx check digit — narrows the shapes a length match cannot prove.
48
+ 4. `listSharedShapeCarrierPatterns()` (`:719`) — shapes several carriers share; the shipping method
49
+ breaks the tie.
50
+ 5. `looksLikeACourierJobNumber()` (`:625`) — the local-courier fallback, gated by
51
+ `hasTrackingNumberPatterns()` (`:638`).
52
+
53
+ ### Certain vs shared shape
54
+
55
+ - **Certain:** UPS `1Z`; USPS `/^9[1-5]\d{20}$/` and `/^9[1-5]\d{24}$/`; the USPS UPU S10
56
+ international forms `/^(EA|EC|CP|RA)\d{9}(\D{2})?$/i` and `/^[A-Za-z]{2}[0-9]+US$/`; Amazon
57
+ `/^:?TBA[0-9]{12,15}$/i`.
58
+ - **Shared shape:** the loose 11-character UPS form, the bare 10-digit form (DHL), and the FedEx digit
59
+ runs.
60
+
61
+ **Every pattern is anchored.** An unanchored 12-digit run matched *inside* free text like
62
+ `INV 986578788855 paid`.
63
+
64
+ Patterns deleted 2026-10-09 as **proven wrong against prod data**: AIT (both patterns — matched one
65
+ SAIA freight row and typed dates such as `11152024`), Dunham (matched nothing; the company closed in
66
+ 2017), the DHL "three letters + seven digits" form (all 8 matching rows were Ingram Micro and New York
67
+ Minute references, and it squatted on Amazon `TBA`), and VYLI's space-padded 19-digit form (it needed
68
+ literal spaces the ODP import trims away). USPS was narrowed from `/^9\d{15,21}$/` because that also
69
+ took `96` and `98` prefixes, which are FedEx — and the tie meant **neither** carrier won. There is **no
70
+ real 14-digit FedEx form**; the 77 Compass rows of that shape are ODP carriers.
71
+
72
+ ### The FedEx check digit does what a pattern cannot
73
+
74
+ 12 digits **is** FedEx's published length, so no pattern can exclude a local courier using the same
75
+ length. Two validators:
76
+
77
+ - `doesFedExTwelveDigitCheckDigitPass()` (`:782`) — digits 1-11 weighted 3,1,7 repeating; sum `MOD 11`
78
+ then `MOD 10` must equal digit 12.
79
+ - `doesFifteenDigitCheckDigitPass()` (`:794`) — the GS1 rule FedEx Ground uses; weights 3,1 walking
80
+ left from the check digit.
81
+
82
+ Measured over **all 13,685** twelve-digit rows in `Client_Compass`: FDEG **2,744 of 2,745** pass, RPSI
83
+ 794 of 794, FDXG 114 of 114 — but VYLI only **104 of 1,371** and SAIA freight **0 of 6**. Prudential:
84
+ **47,462 of 47,469**. So the check digit is exactly what separates real FedEx from Veyer and LTL
85
+ freight. **A carrier with no rule defined always passes**, so this only ever narrows FedEx.
86
+
87
+ ### Edge punctuation only — middles are left alone deliberately
88
+
89
+ 9 real `Client_Nychh` rows are a valid FedEx number with a leading colon (e.g. `:557705166169`), and
90
+ the patterns are anchored, so one stray edge character hides a good number. Middle punctuation stays
91
+ untouched so `2848880,2849200` and `512181and3174` remain unmatched.
92
+
93
+ ### Local couriers resolve through the shipping method, not a pattern
94
+
95
+ A local courier must be a **database row**, never a pattern in shared library code — the per-client
96
+ carrier signal is the `ShippingMethods` row (see below), not the number.
97
+
98
+ `looksLikeACourierJobNumber()` accepts a **5 to 9 digit** run (spaces and dashes allowed). The 9-digit
99
+ cap is what stops a client carrier code such as `FDEG` from claiming a 12-digit number when the name
100
+ check misses it. `hasTrackingNumberPatterns($carrierName)` returns true when a carrier could ever be
101
+ named by a number alone; if so, the shipping method is **not** allowed to override a blank result — a
102
+ real FedEx number would have matched above, so the method is lying.
103
+
104
+ NYCHH is the first client wired this way: `dbchanges2/Client_Nychh/2026-10-09 - NewYorkMinuteShippingCarrier.sql`
105
+ adds carrier **New York Minute** (uuid `6f3c8d47-21ba-4e95-b70d-5c91a4f8e2d3`) and repoints
106
+ `ShippingMethods` id 5 (`c_netsuiteInternalShipMethodId = 33322`) onto it. The blank-named carrier id 12
107
+ is **deliberately left alone** — the sync reuses it as a placeholder.
108
+
109
+ ### The shipping method is the only per-client carrier signal
110
+
111
+ NetSuite has **no usable carrier field** on an item fulfillment: `transaction.shipcarrier` is the
112
+ literal grouping string `FedEx/USPS/More` on every row, and a `shipitem` record has no carrier column.
113
+ The only signal is the TOGa `ShippingMethods` row matched on `c_netsuiteInternalShipMethodId`.
114
+ `App_Api_Netsuite_Rest::fetchItemFulfillmentById` therefore exposes `shipMethod` (internalId + name) on
115
+ its shim — see [the REST shim doc](./netsuite-suiteql-rest-shim.md).
116
+
117
+ ## Gotchas / known issues
118
+
119
+ - **⚠ api2 treats `tracking-numbers.shippingCarrierId` as a MATCH field.** Sending a carrier name the
120
+ client has no `ShippingCarriers` row for does **not** fall back to NULL — it fails the **whole**
121
+ tracking-number write with *unable to find a unique match*. The sync therefore only ever sends a
122
+ carrier the client already has, and **never creates one**. Auto-creating carriers is what produced
123
+ 340 duplicate `ShippingCarriers` rows across 14 clients (TRUE-81576); see
124
+ [shipping carriers & accounts](../../../../2.0/apps/_underscore/features/shipping-carriers-and-accounts.md).
125
+ - **⚠ KNOWN GAP — `hasTrackingNumberPatterns()` compares the carrier NAME**, so client-specific codes
126
+ (`FDEG`, `FEDX`, `UPSG`) defeat it. Only the 9-digit cap bounds the damage. The proper fix is a
127
+ local-courier flag driven through settings, **not** a new field. Not built yet.
128
+ - **A bare 10-digit number must NOT resolve to DHL on its own (decided 2026-10-09).** A CodeRabbit
129
+ review asked `generateTrackingNumberLink()` / `displayTrackingNumberWithCarrier()` to fall back to
130
+ "exactly one shared-shape match" when no shipping method is passed. **Rejected:** a bare 10-digit
131
+ number matches only the DHL pattern (shared UPS needs 11), so "exactly one match" *always* means DHL.
132
+ Prod 10-digit rows: Compass 25 FedEx, 3 FXFE, 1 Tripp Lite against only 3 DHL; NYCHH 57 labelled DHL
133
+ although NYCHH never ships DHL; Prudential 2. **Of 103 rows, 5 are really DHL.** The wrong NYCHH
134
+ labels came from the old loose DHL pattern in the 2.0 route
135
+ (`_underscore/Trait/Netsuite/ItemFulfillment.php`). Accepted cost: 5 rows lose the `(DHL)` label and
136
+ fall back to the Google-search link `generateTrackingNumberLink()` already uses for unknown numbers.
137
+ - **A vendor-feed import already knows its carrier — do not re-guess it.** The Compass ODP and Compass
138
+ Canada G&T ASN crons take tracking numbers from the vendor feed, which carries the real carrier code
139
+ (`VYLI`, `FDEG`, `SAIA`, `RPSI`), and the import maps it into `shippingCarrierId` at
140
+ `worker/crons/toga2/compass/workflow/3b_import_strategic_systems_advance_shipping_notices.php:495`.
141
+ They call `getCarrierFromTrackingNumber()` only afterwards, for the carrier name in the email link,
142
+ and need **no** NetSuite shipping method. Follow-up: read the name off the row just saved.
143
+ - **NYCHH has 860 tracking numbers in the 5-9 digit job-number shape** (sample `001056`), all with no
144
+ carrier today. They only get New York Minute on **future** syncs unless someone backfills.
145
+
146
+ ## Testing
147
+
148
+ **197 tests** across three suites (patterns, check digits, shipping-method fallback), all passing.
149
+
150
+ Measured regression on NYCHH, 2,952 tracking numbers that have a carrier today: **2,869 keep the same
151
+ carrier**, 58 DHL and 25 FedEx go **blank**, and **none** move to a wrong carrier.
152
+
153
+ The three USPS shapes the old code caught and the new one does not (short `9x` numbers of 16-21 digits,
154
+ the `82` + 8-digit form, and `E` + 9 digits that is not EA/EC/CP/RA) return **zero rows across the 12
155
+ biggest prod clients** — so the ~20 worker crons that call `getCarrierFromTrackingNumber()` for email
156
+ links are unaffected.
157
+
158
+ ## Change history
159
+ - 2026-10-09 — **Built.** Carrier is now set on tracking numbers during the NetSuite item-fulfillment
160
+ sync (TRUE-82527): new `findShippingCarrierForTrackingNumber()`, patterns split into certain vs
161
+ shared-shape with the wrong ones deleted, FedEx check-digit validation, edge-punctuation stripping,
162
+ and a shipping-method-driven local-courier fallback (NYCHH New York Minute). Decided that a bare
163
+ 10-digit number must not resolve to DHL. (bala)
164
+
165
+ ## Related
166
+ - [NetSuite SuiteQL/REST shim](./netsuite-suiteql-rest-shim.md) — the `shipMethod` field, and why
167
+ `shipcarrier` is useless
168
+ - [NetSuite → TOGa Supply per-client sync](../../worker/features/netsuite-togasupply-per-client-sync.md)
169
+ - [Shipping carriers & accounts](../../../../2.0/apps/_underscore/features/shipping-carriers-and-accounts.md)
170
+ - [NYCHH transfer-order emails](../../../../clients/nychh/features/transfer-order-notification-emails.md)
@@ -6,14 +6,15 @@ project: Library
6
6
  client: shared
7
7
  type: feature
8
8
  status: active
9
- updated: 2026-09-21
10
- owners: [dfranks, jcardinal, kyalamarthi, rgirish]
9
+ updated: 2026-10-09
10
+ owners: [dfranks, jcardinal, kyalamarthi, rgirish, bala]
11
11
  files:
12
12
  - library/app/api/netsuite/rest.php
13
13
  - library/app/netsuite.php
14
14
  - worker/crons/toga2/forecast2/common_import_sales_from_netsuite.php
15
15
  related:
16
16
  - netsuite-suiteql-api-reference.md
17
+ - ./carrier-detection-from-tracking-number.md
17
18
  - ../architecture.md
18
19
  - ../../worker/features/forecast2-netsuite-reconciliation.md
19
20
  - ../../../2.0/apps/_underscore/features/netsuite-rest-client.md
@@ -54,6 +55,8 @@ Header SuiteQL → per-record detail → line SuiteQL, assembled into SOAP-shape
54
55
 
55
56
  Field semantics (verified on the live account):
56
57
 
58
+ - **⚠ There is NO usable carrier field on an item fulfillment — read the ship method instead.** `transaction.shipcarrier` is the literal grouping string `FedEx/USPS/More` on **every** row, and a `shipitem` record has no carrier column. So `fetchItemFulfillmentById` now exposes **`shipMethod`** on the shim (internalId + name, read from the REST GET body like opportunity / terms / createdFrom), and the carrier is resolved on the TOGa side from the `ShippingMethods` row matched on `c_netsuiteInternalShipMethodId`. Anything needing a carrier reads that, never `shipcarrier`. See [carrier detection from a tracking number](./carrier-detection-from-tracking-number.md).
59
+
57
60
  - **Line number = `transactionline.id`, NOT `linesequencenumber`.** `tl.id` equals the REST `item[].line` value (the stable SOAP-era line key); `linesequencenumber` is physical order and re-maps/collides when SOAP-era records are edited. Verified: sales orders 106 orders / 344 lines (2019–2029, incl. edited orders with line-id gaps), invoices + cash sales 62/62. `listSales` historically resolved line via `uniquekey → REST lineUniqueKey → line`; `tl.id` gives the same answer in bulk without the per-id GET. **Re-confirmed at scale 2026-08-27:** the two columns diverge on **2,061 of 4,387** lines (~47%), and `Forecast.OpenOrderItems.lineNumber` (which stores REST `$line->line`) matched `transactionline.id` on **99 of 99** randomly sampled prod rows plus all 7 multi-root orders, with zero join misses. So keying a NetSuite ↔ Forecast line join on `linesequencenumber` silently mismatches about half the book — always join on `transactionline.id`.
58
61
  - **ShipItem lines have `costestimate = NULL`** (100% of 42K+ lines, 2025–26). In any profit expression `SUM(-foreignamount + costestimate)`, a NULL cost makes the whole term NULL and SQL **drops the row from the SUM** — so shipping *revenue* still counts but shipping *profit* does not, silently understating NS profit. Always `NVL(tl.costestimate, 0)`. (Caused a fake $663K "gap" in the audit tools — see the reconciliation doc.)
59
62
  - **Cash refunds & credit memos carry extra COGS/inventory lines.** SuiteQL `transactionline` with `mainline='F' AND item>0` returns, per item, the customer line **plus** a `CUSTOMERRETURN` and an `ASSET` posting (all `item>0`), which REST's `item[].items` sublist omits. They are distinguished by **`tl.iscogs = 'T'`** (and `accountinglinetype` IN `CUSTOMERRETURN`/`ASSET`). Filter **`tl.iscogs = 'F'`** to get exactly the customer-facing lines. Invoices/cash sales do not have this (0 extra lines). Without the filter, bulk imports duplicate refund/memo lines.
@@ -95,6 +98,7 @@ Field semantics (verified on the live account):
95
98
  - **`App_NetSuite::getCustomFieldValue()` (`library/app/netsuite.php`) must be warning-safe.** 1.0 runs on PHP 7.2 in an unguarded path, so any PHP warning → `exit()` (kills the cron). It was hardened 2026-08-27 with `?? []` / `?? null` guards on the custom-field-list traversal. Keep that discipline when touching it.
96
99
 
97
100
  ## Change history
101
+ - 2026-10-09 — `fetchItemFulfillmentById` now exposes `shipMethod` (internalId + name), because `transaction.shipcarrier` is the useless literal `FedEx/USPS/More` on every row and `shipitem` has no carrier column. It feeds the 1.0 carrier resolver. (bala)
98
102
  - 2026-09-21 — `listSales()` gained a `$recordTypes` allowlist param (defaults to all 4 for the forecast import); callers pass `RECORD_TYPE_*` constants, not raw SuiteQL codes; throws on unknown/empty. Lets the togasupply sync request INVOICES only. (jcardinal)
99
103
 
100
104
  ## Related
@@ -6,7 +6,7 @@ project: Worker
6
6
  client: shared
7
7
  type: feature
8
8
  status: active
9
- updated: 2026-10-06
9
+ updated: 2026-10-09
10
10
  owners: ["dfranks", "bala", "jcardinal", "mhammontree", "snaredla", "rgirish"]
11
11
  files:
12
12
  - worker/crons/toga2/netsuite/common_sync_togasupply.php
@@ -35,6 +35,7 @@ related:
35
35
  - ../../../clients/elite/features/netsuite-togasupply-sync.md
36
36
  - ../../library/features/netsuite-item-class-sync.md
37
37
  - ../../library/features/netsuite-item-description-sync.md
38
+ - ../../library/features/carrier-detection-from-tracking-number.md
38
39
  - ../../../clients/compass-usa/features/sales-order-line-renumbering.md
39
40
  - ../../../2.0/apps/_underscore/features/acl-permission-chain.md
40
41
  - ../../../2.0/apps/_underscore/features/sales-order-purchase-order-bridge-direction.md
@@ -751,6 +752,19 @@ the [Quad profile](../../../clients/quad/profile.md) and the
751
752
  Design rationale for putting this at the source writer rather than a 317 interceptor lives on the
752
753
  [recursive item fulfillments doc](../../../2.0/apps/_underscore/features/recursive-item-fulfillments.md).
753
754
 
755
+ ### The fan-out now also sets the CARRIER on the tracking number (2026-10-09)
756
+
757
+ `syncItemFulfillmentFromNetsuite` used to write package tracking numbers with **no**
758
+ `shippingCarrierId`, so every screen and email showed a bare number with no carrier and no tracking
759
+ link. It now reads the fulfillment's **shipping-method carrier once per fulfillment** (from the shim's
760
+ new `shipMethod` field — NetSuite has no carrier field worth reading) and passes it to
761
+ `App_Misc::findShippingCarrierForTrackingNumber()`, which decides from the **number first** and uses
762
+ the method only as a fallback.
763
+
764
+ **It never creates a carrier row.** `tracking-numbers.shippingCarrierId` is an api2 MATCH field, so a
765
+ name the client has no row for fails the **whole** tracking-number write. Full rules, patterns and
766
+ measured results: [carrier detection from a tracking number](../../library/features/carrier-detection-from-tracking-number.md).
767
+
754
768
  ### Fulfillment line → sales-order line matching is a 4-step chain (2026-09-13)
755
769
 
756
770
  The fulfillment matcher in `syncItemFulfillmentFromNetsuite` resolves each fulfillment line back
@@ -2126,6 +2140,7 @@ library (or vice versa) crashes GroWrk and Adyen on their next sync run.
2126
2140
  [NetSuite Sync Alert Monitor](../../library/features/netsuite-sync-alert-monitor.md).
2127
2141
 
2128
2142
  ## Change history
2143
+ - 2026-10-09 — **Item-fulfillment tracking numbers now carry a carrier** (TRUE-82527). The fan-out passes the fulfillment’s shipping-method carrier into ; the number decides, the method is only a fallback, and no carrier row is ever created. (bala)
2129
2144
  - 2026-10-06 — Added "Fix rules" section: self-heal, repair-by-code, fail-loud, dependencies-first, no forward cursor skip, links-are-data, Compass delete boundary. (jcardinal)
2130
2145
  - 2026-10-06 — NYCHH ITEM_FULFILLMENTS frozen 09-30→10-06 (Event issue 483, FK 1451 on IA→IF line link) fixed: reverse retire TO→SO + IA-link move before every IF-line delete (library only). Corrected "NYCHH once a day" — it runs every 5 min. (jcardinal)
2131
2146
  - 2026-10-05 — Closed-order delete now per-client SO scoped (Compass dual-SO freeze, Issue 940); details in [closed-order rule](../../library/features/netsuite-closed-order-rule.md). Library only. (jcardinal)
@@ -6,8 +6,8 @@ project: _Underscore
6
6
  client: shared
7
7
  type: feature
8
8
  status: active
9
- updated: 2026-09-16
10
- owners: [mhammontree]
9
+ updated: 2026-10-09
10
+ owners: [mhammontree, bala]
11
11
  files:
12
12
  - _underscore/Trait/Netsuite/SalesOrder.php
13
13
  - _underscore/Component/Library/Carriers/Fedex/Fedex.php
@@ -24,6 +24,7 @@ related:
24
24
  - ../../toga2-supply/features/fulfill-and-ship.md
25
25
  - ../../../1.0/apps/worker/features/netsuite-togasupply-per-client-sync.md
26
26
  - ../../../clients/growrk/features/shipping-carrier-cleanup.md
27
+ - ../../../../1.0/apps/library/features/carrier-detection-from-tracking-number.md
27
28
  ---
28
29
 
29
30
  Per-client `ShippingCarriers` / `ShippingCarrierAccountNumbers` model, how the syncs match/create carrier rows, and the duplication + blank-carrier + pagination traps; open before touching carrier sync, the Fulfill & Ship dropdown, or deleting a carrier.
@@ -57,6 +58,16 @@ The Fulfill & Ship **Carrier dropdown only lists carriers with at least one `Shi
57
58
 
58
59
  **Fix:** a paged `GET /shipping-carriers` loop that **seeds the lookup directly**, using the same `for ($page = 1, $nextPage = 0; ...)` idiom and `MAX_RECORDS_PER_PAGE_TO_FETCH_FROM_TOGA_API` (1000) as the neighbouring lookups. Blank names are skipped — they are unusable as keys.
59
60
 
61
+ ### ⚠ NEVER auto-create a carrier — and NEVER send a carrier name the client does not have
62
+
63
+ `tracking-numbers.shippingCarrierId` is a **MATCH field** in api2. Sending a carrier name with no
64
+ `ShippingCarriers` row for that client does **not** degrade to NULL — it fails the **whole**
65
+ tracking-number write with *unable to find a unique match*. So a writer has exactly two safe moves:
66
+ send a carrier the client already has, or send none. **Creating the missing row is what produced the
67
+ duplicates above** (340 rows across 14 clients). The 1.0 carrier-detection path follows this rule as of
68
+ 2026-10-09: it looks the carrier up and sends nothing when there is no row. See
69
+ [carrier detection from a tracking number](../../../../1.0/apps/library/features/carrier-detection-from-tracking-number.md).
70
+
60
71
  ### A blank-named carrier is a WILDCARD (PHP stripos with an empty needle)
61
72
 
62
73
  The carrier matcher keys on `stripos($nsShippingMethodName, $carrierName) === 0`. Under PHP 8 an **empty needle returns `int(0)`** — verified directly:
@@ -96,11 +107,18 @@ Before deleting a `ShippingCarriers` row in a client DB, check **all** of:
96
107
  ## Gotchas / known issues
97
108
 
98
109
  - **Cleaning duplicates before the fix is deployed is wasted work** — the syncs refill the rows on the next run (every 5 minutes). Deploy first, clean second.
99
- - Compass (98 carriers) and NYC Health & Hospitals (100) are **not** cleaned yet — separate tasks.
110
+ - **CLEANED. Swept all 37 prod client DBs on 2026-10-09: zero duplicate carriers remain and zero `ShippingMethods` rows have a NULL `shippingCarrierId`.** The merge collapsed **340** duplicate rows across **14** clients. **The merge needed a Part 1 that merged duplicate `ShippingMethods` first** — merging carriers alone hits `Duplicate entry '6-UPS' for key 'ShippingMethods.shippingCarrierId_name'`. ⚠ The merge SQL is **not in dbchanges2 on any branch** (checked 2026-10-09) — it was run ad-hoc, so re-create it if the merge has to be repeated anywhere.
111
+ - **A blank-named carrier row is NORMAL, not damage.** Eight clients have exactly one (Canon id 5, Compass 23, Endeavorhealth 5, Growrk 29, Nychh 12, Prudential 6, Spglobal 60, Wmchealth 6). Each is attached to 1-5 shipping methods and to **zero** tracking numbers — it is the placeholder the sync reuses (see above). **Do not delete it** and do not repoint its methods blindly; give the method a real carrier instead, the way NYCHH's method id 5 was repointed onto *New York Minute*.
100
112
  - The `[fedex]` config group contains a key literally named **`_`** with a 32-character value. Purpose unknown; no reader has been traced.
101
113
 
114
+ ## Change history
115
+ - 2026-10-09 — **Duplicate carriers are gone** (sweep of all 37 prod client DBs) and the blank-named
116
+ placeholder row is documented as expected. Added the rule that `tracking-numbers.shippingCarrierId`
117
+ is an api2 MATCH field, so a writer must never invent a carrier. (bala)
118
+
102
119
  ## Related
103
120
 
121
+ - [carrier detection from a tracking number](../../../../1.0/apps/library/features/carrier-detection-from-tracking-number.md)
104
122
  - [carrier-shipping-labels](carrier-shipping-labels.md)
105
123
  - [address-validation](address-validation.md)
106
124
  - [fulfill-and-ship](../../toga2-supply/features/fulfill-and-ship.md)
@@ -4,7 +4,7 @@ _Auto-generated by `knowledge.js index`. Do not hand-edit._
4
4
 
5
5
  ## 1.0 framework
6
6
 
7
- - **library** (Library) _(framework core)_ — 34 doc(s) → [1.0/apps/library/INDEX.md](1.0/apps/library/INDEX.md)
7
+ - **library** (Library) _(framework core)_ — 35 doc(s) → [1.0/apps/library/INDEX.md](1.0/apps/library/INDEX.md)
8
8
  - **worker** (Worker) — 42 doc(s) → [1.0/apps/worker/INDEX.md](1.0/apps/worker/INDEX.md)
9
9
  - **dbchanges** (Database Changes) _(framework core)_ — 1 doc(s) → [1.0/apps/dbchanges/INDEX.md](1.0/apps/dbchanges/INDEX.md)
10
10
  - **worker1.5** (Worker 1.5) — 0 doc(s) → [1.0/apps/worker1.5/INDEX.md](1.0/apps/worker1.5/INDEX.md)
@@ -6,8 +6,8 @@ project: Worker
6
6
  client: compass-usa
7
7
  type: client-feature
8
8
  status: active
9
- updated: 2026-09-22
10
- owners: ["rgirish"]
9
+ updated: 2026-10-09
10
+ owners: ["rgirish", "bala"]
11
11
  files:
12
12
  - worker/crons/toga2/compass/workflow/3b_import_strategic_systems_advance_shipping_notices.php
13
13
  - worker/crons/toga2/compass/DOA_Scripts/DOA_compass_statuses.php
@@ -18,6 +18,7 @@ related:
18
18
  - ../../compass-canada/features/grand-and-toy-asn-import.md
19
19
  - ../../quad/features/po-asn-email-import.md
20
20
  - ../../../1.0/apps/worker/architecture.md
21
+ - ../../../1.0/apps/library/features/carrier-detection-from-tracking-number.md
21
22
  ---
22
23
  The 1.0 worker cron that reads Office Depot / Strategic Systems ASN CSVs from email, why it routes by column count, and why a tracking number can arrive as `8.77536E+11` with its digits already gone. Open before touching tracking-number parsing in this file or its DOA copy.
23
24
 
@@ -59,6 +60,15 @@ Both files parse tracking numbers in two branches each — ODP and Strategic Sys
59
60
 
60
61
  ## Gotchas
61
62
 
63
+ - **The carrier on an ODP / Strategic Systems row comes from the FEED, not from the tracking number.**
64
+ The import maps the feed’s carrier code (`VYLI`, `FDEG`, `SAIA`, `RPSI`) into `shippingCarrierId` at
65
+ `3b_import_strategic_systems_advance_shipping_notices.php:495`; `App_Misc::getCarrierFromTrackingNumber()`
66
+ runs only **afterwards**, for the carrier name in the email link. So this cron needs **no** NetSuite
67
+ shipping method (same for the Compass Canada G&T ASN cron). **Follow-up:** read the name off the row
68
+ just saved instead of re-guessing — the number cannot tell VYLI from FedEx (only 104 of 1,371 VYLI
69
+ numbers pass the FedEx check digit). See
70
+ [carrier detection from a tracking number](../../../1.0/apps/library/features/carrier-detection-from-tracking-number.md).
71
+
62
72
  - **⚠ A scientific-notation tracking number is a DATA bug, not a display bug — and the digits are gone before our code runs.** `Client_Compass.TrackingNumbers.number` literally stores the 11-character string `8.77536E+11` (e.g. id 80112, `shippingCarrierId` NULL). 44 such rows exist, Sep 2025 → 2026-09-22. The cause is upstream: a 12-digit number like `877536271001` becomes `8.77536E+11` when a spreadsheet saves the column as a number rather than text. Only 6 significant digits survive. Reported on order **SA137640**.
63
73
 
64
74
  - **⚠ NEVER "recover" a scientific-notation number with `number_format()` — it pads zeros and fabricates a believable-but-wrong tracking number.** A guard added 2026-10-17 (commit `be8ff1d14`) ran `number_format($n, 0, '', '')` on any value containing `e+`. Verified by running it: `number_format("8.77536E+11")` returns **`877536000000`**, while the real value was **`877536271001`**. The result is a 12-digit number that looks perfectly valid, so nobody can spot it as fake — strictly worse than failing the row. Removed 2026-09-22; all four sites now reject instead. **A silent wrong tracking number is worse than a loud rejected row.**
@@ -80,5 +90,7 @@ Both files parse tracking numbers in two branches each — ODP and Strategic Sys
80
90
  - The Compass **Canada** G&T importer already rejects scientific-notation tracking as a row-level guard; its read-only inbox inspector reports it as a dry-run category. That is the pattern Compass USA now matches.
81
91
 
82
92
  ## Change history
93
+ - 2026-10-09 — Recorded that the feed, not the tracking number, is the carrier source here; the 1.0
94
+ pattern rewrite does not change what this cron stores. (bala)
83
95
  - 2026-09-22 — Removed the lossy `number_format()` "recovery" from all 4 guard sites in both files; now split-then-guard, appending to `$validationErrors` instead. Fixes the fabricated-tracking-number bug and the multi-value PHP fatal. Written and linted (`php -l` passes), **not deployed**. (rgirish)
84
96
  - 2026-09-22 — Documented CSV-vs-XLSX as unrecoverable-at-source, column-count file routing, the unknown 06:22 writer, and the 168-row data-quality survey. (rgirish)
@@ -6,7 +6,7 @@ project: Worker
6
6
  client: nychh
7
7
  type: client-feature
8
8
  status: active
9
- updated: 2026-10-05
9
+ updated: 2026-10-09
10
10
  owners: ["bala", "rgirish", "jcardinal"]
11
11
  files:
12
12
  - worker2/Worker/Client/Nychh/TransferOrderEmails.php
@@ -17,6 +17,7 @@ files:
17
17
  - dbchanges2/Client_Nychh/2026-09-15b - TransferOrderEmailGoLiveRecipient.sql
18
18
  - dbchanges2/Core/2026-09-16a - NychhTransferOrderEmailsCron.sql
19
19
  - dbchanges2/Client_Nychh/2026-10-05a - TransferOrderShippedEmailOnTrackingCutover.sql
20
+ - dbchanges2/Client_Nychh/2026-10-09 - NewYorkMinuteShippingCarrier.sql
20
21
  related:
21
22
  - ./transfer-order-netsuite-push.md
22
23
  - ../../../2.0/apps/_underscore/features/units-for-items-for-purchase-orders.md
@@ -29,6 +30,7 @@ related:
29
30
  - ../../../2.0/apps/worker2/features/creating-worker-actions.md
30
31
  - ../../../2.0/apps/worker2/features/sandbox-dev-worker-environments.md
31
32
  - ../profile.md
33
+ - ../../../1.0/apps/library/features/carrier-detection-from-tracking-number.md
32
34
  ---
33
35
 
34
36
  ## Summary
@@ -207,7 +209,7 @@ Added 2026-09-30 (TRUE-82111). `SendFulfillment` runs `readLinesMissingAssetTags
207
209
 
208
210
  - the item is **`SERIALIZED`** and it has **fewer linked units than its quantity**
209
211
  (`unitsLinked < ItemFulfillmentItems.quantity`), or
210
- - a linked unit has a **blank `Units.assetTag`** — that half holds whatever the item type is.
212
+ - a linked unit has a **blank `Units.assetTag`** — SERIALIZED or HYBRID only since 2026-10-09.
211
213
 
212
214
  **⚠ The unit-count half is SERIALIZED-only.** `Client_Nychh.Items.inventoryType` is
213
215
  `enum('SERIALIZED','NON_SERIALIZED','HYBRID')`; only SERIALIZED items are expected to carry units,
@@ -221,6 +223,12 @@ passed at 10:10 and Order Packed went out with the accessory line correctly igno
221
223
  underlying rule lives in
222
224
  [inventoryType determines unit existence](../../../2.0/apps/_underscore/features/units-for-items-for-purchase-orders.md).
223
225
 
226
+ **⚠ NON_SERIALIZED is now excluded from BOTH reads (2026-10-09).** `readLinesMissingAssetTags()`
227
+ (the blank-`assetTag` half, `:354`) **and** `readUnitsForEmail()` (`:676`, the asset-tag table printed
228
+ in the customer email) both skip `NON_SERIALIZED`. Item **580-BBQL** had been set up as serialized in
229
+ NetSuite by mistake, so the email rendered an **empty asset-tag table** and ops got a **false**
230
+ exception email. A mis-typed item in NetSuite must not produce either.
231
+
224
232
  When any line is short, `SendFulfillment` **returns before writing
225
233
  `c_transferOrderEmailStageSent`** — so packed stays pending and the backstop cron re-sends it by
226
234
  itself the moment the tags arrive. **The hold covers shipped too**, because the early return happens
@@ -307,6 +315,17 @@ Cost of the split, stated so nobody is surprised later:
307
315
  when there is no number and drops the button when there is no URL (since 2026-10-05 Shipped needs a
308
316
  number, so this now matters only for the Packed email). Placeholders are
309
317
  **`{trackingIntroSuffix}`** and **`{trackingSection}`**.
318
+ - **Carrier logos are a hardcoded map, because `ShippingCarriers.urlLogo` is an app path an inbox
319
+ cannot open.** `CARRIER_LOGO_URL_BY_CARRIER_NAME` (`:46`) keys on the **uppercased** carrier name,
320
+ which is read by joining `ShippingCarriers` on `TrackingNumbers.shippingCarrierId` (`:701`). Added
321
+ 2026-10-09: **`NEW YORK MINUTE`** → `NYMinuteMessenger_Logo_2022_4C.png` on `toga-public`. **New York
322
+ Minute has no tracking website**, so that email shows the logo plus the plain number and **no link**.
323
+ The carrier reaches the number through the ship method — `ShippingMethods` id 5
324
+ (`c_netsuiteInternalShipMethodId = 33322`) was repointed onto the new carrier by
325
+ `dbchanges2/Client_Nychh/2026-10-09 - NewYorkMinuteShippingCarrier.sql`; see
326
+ [carrier detection from a tracking number](../../../1.0/apps/library/features/carrier-detection-from-tracking-number.md).
327
+ **⚠ 860 existing NYCHH numbers in the 5-9 digit job-number shape still have no carrier** (sample
328
+ `001056`) — only future syncs fix them unless someone backfills.
310
329
  - **Gmail clips a message past about 102,400 bytes.** `MAXIMUM_EMAIL_BODY_BYTES = 80000` budgets the
311
330
  **item block only**; the remainder is summarised as *"and N more"*. 80,000 (not the earlier
312
331
  92,000) because the stored shipped template is **11,695 bytes** on its own and a 25-package
@@ -396,6 +415,12 @@ loaded out of the dbchanges2 file so an unresolved `{placeholder}` fails a test.
396
415
  unescaped values and both detail-box defects were all found there, not in production.
397
416
 
398
417
  ## Change history
418
+ - 2026-10-09 — **New York Minute shows its logo, and NON_SERIALIZED lines are ignored by the asset-tag
419
+ reads** (TRUE-82527). `CARRIER_LOGO_URL_BY_CARRIER_NAME` gained `NEW YORK MINUTE` (no tracking site,
420
+ so logo + plain number, no link), paired with `Client_Nychh/2026-10-09 - NewYorkMinuteShippingCarrier.sql`
421
+ and the new 1.0 carrier resolver. `readUnitsForEmail()` and `readLinesMissingAssetTags()` now exclude
422
+ `NON_SERIALIZED`, after item 580-BBQL (mis-set as serialized in NetSuite) produced a blank asset-tag
423
+ table and a false exception email. (bala)
399
424
  - 2026-10-05 — **Shipped now fires on the first tracking number, not the NetSuite shipped stage**
400
425
  (warehouse marks shipped after delivery). Shared `buildHasTrackingNumberSql()`, new
401
426
  `resolveStagesStillToAnnounce()` table, matching `SendPending` WHERE; removed dead
@@ -49,7 +49,9 @@ writes, and agents until the session has a ticket. These replaced `/kickoff`, `/
49
49
  4. **`/finish`** — always writes the team KB first. Then one approval list for ticket fields
50
50
  (Application, Stakeholders, Task Type, title, Has UI Changes). If the ticket is done: push each
51
51
  repo with work, open or update a PR per repo, post a PM comment, set status
52
- `development finished`.
52
+ `development finished`. The PM comment is plain bullets for a non-technical PM: "What
53
+ changed" always; "Manual steps", "Still open", "Ticket fields changed by Claude" only when
54
+ they have content. No Apps/Clients/PRs sections. Template: `skills/finish/SKILL.md` Step 15.
53
55
 
54
56
  ### Ticket branch at /start
55
57
 
@@ -164,6 +166,8 @@ All tested 2026-10-08:
164
166
  `development finished` only. (jcardinal)
165
167
  - 2026-10-09 — TRUE-82605: `clickup_update_task` and `clickup_create_comment` no longer ask;
166
168
  the gate is the limit, field changes need the dev's yes in chat. (jcardinal)
169
+ - 2026-10-09 — PM feedback: PM comment is plain bullets for a non-technical PM; Apps, Clients,
170
+ PRs removed; empty sections left out; field changes as plain sentences with the reason. (jcardinal)
167
171
 
168
172
  ## Related
169
173
 
package/package.json CHANGED
@@ -1,6 +1,6 @@
1
1
  {
2
2
  "name": "toga-ai",
3
- "version": "1.0.987",
3
+ "version": "1.0.989",
4
4
  "description": "TOGA Technology Team Claude Knowledge System — shared AI coding harness with skills, knowledge base CLI, and project installer for Claude Code.",
5
5
  "keywords": [
6
6
  "claude",
@@ -482,7 +482,7 @@ No session notes at all → under WHY write `not recorded — this work was not
482
482
  Never copy a key, token, or password into the body — write `a secret (redacted)` and list it
483
483
  under TEAM RULE CHECK.
484
484
 
485
- Keep every PR url for Steps 15 and 17.
485
+ Keep every PR url for Step 17.
486
486
 
487
487
  **Secret scan** (PR body and PM comment, before posting):
488
488
 
@@ -496,19 +496,43 @@ could not be read — fix the path. Never post until it prints `{ok:true}`.
496
496
 
497
497
  ## Step 15 — PM comment (Claude writes and posts it — no approval)
498
498
 
499
- Draft a short comment for the PM and release notes. Plain words, no code dumps, no secrets,
500
- no local paths:
499
+ Draft a short comment for a non-technical project manager (also used for release notes):
501
500
 
502
501
  ```
503
- **What changed:** <for users/clients, 1-3 plain lines>
504
- **Apps:** <Application values>
505
- **Clients:** <Stakeholders values>
506
- **PRs:** <one url per repo>
507
- **Manual steps:** <deploy / SQL file + database + order — or "None">
508
- **Still open:** <anything left — or "Nothing">
509
- **Ticket fields changed by Claude:** <from Step 10 — or "None">
502
+ **What changed**
503
+
504
+ - <plain, PM-friendly point>
505
+ - <...>
506
+
507
+ **Manual steps**
508
+
509
+ - <plain point>
510
+
511
+ **Still open**
512
+
513
+ - <plain point>
514
+
515
+ **Ticket fields changed by Claude**
516
+
517
+ - <plain sentence with the reason>
510
518
  ```
511
519
 
520
+ Rules:
521
+
522
+ 1. **Write for a PM, not a developer.** Say what users or clients will notice, or what the work
523
+ achieved. No file names, paths, commands, skill names, code terms, code dumps, or secrets.
524
+ 2. **Bullets in every section.** One point per bullet.
525
+ 3. **Only "What changed" is always there.** Leave out "Manual steps", "Still open", and
526
+ "Ticket fields changed by Claude" entirely when they have nothing — never write "None".
527
+ 4. **Manual steps in plain words** — e.g. "A database update must run before release", not a
528
+ SQL file name.
529
+ 5. **Ticket field changes (from Step 10) as plain sentences with the reason** — e.g. "The Has
530
+ UI Changes field was unchecked because this task had no UI changes." / "Added TOGa Desk to
531
+ Application because the work also changed the Desk app."
532
+ 6. **No Apps, Clients, or PRs sections.** Apps and clients are in the ticket fields; PRs link
533
+ to the ticket through the GitHub integration.
534
+ 7. **One blank line between sections**, and between a heading and its bullets.
535
+
512
536
  Do not ask the developer to approve it. Write the text to a temp file
513
537
  (`toga-comment-<TICKET>.md` in the OS temp dir), run the **Secret scan** on it, delete the
514
538
  file. Stop only on a hit. Then post it, and show the posted text in the Step 17 report: