toga-ai 1.0.625 → 1.0.626

package/knowledge/clients/nycdoe/features/hold-status-sync.md

@@ -6,7 +6,7 @@ project: Worker
 client: nycdoe
 type: client-feature
 status: active
-updated: 2026-07-20
+updated: 2026-08-20
 owners: [mhammontree, sking]
 files:
   - worker/crons/sync/nycdoe/send_ticket_updates.php
@@ -76,48 +76,87 @@ revert or never reach ServiceNow. It is the status-sync companion to the broader
 - `Hold` has **statusRank 6**, beating `Work in Progress` (rank 5) in the downgrade-guard
   added by TRUE-76812 — so a downstream sync cannot silently demote a held order.
 
-### Status fields (ServiceNow side) — CORRECTED 2026-07-20
-> **The earlier "native `state` is authoritative; never use `u_status_task`" model was WRONG**
-> (see Change history 2026-07-20). Both fields exist and both matter, with distinct roles:
-- **`u_status_task` (custom) is the CLIENT-FACING, authoritative incident status** — the field
-  DOE's own ServiceNow views/reporting read (values `Open`, `In Progress`, `On Hold`, …). **A hold
-  MUST be written here or the client never sees it.** It is **writable from any active state**; a
-  controlled test PATCH of `{u_status_task:'On Hold'}` returned success and PERSISTED for days
-  (INC2192842 held 3+ days). **The TOGaDesk inbound sync keys off `u_status_task`**
-  (`process_tickets.php:206` reads `$data->u_status_task`), and the RITM cron checks
-  `u_status_task == 'On Hold'` (`send_request_item_updates.php:224`) — so `u_status_task` **is**
-  the correct field for hold detection.
-- The native **`state`** field is ServiceNow's INTERNAL, SLA-bearing state machine. Setting
-  `state=3` (On Hold) pauses the SNOW SLA clock but is **invisible to the DOE client**. ServiceNow
-  only permits the native On Hold transition **FROM In Progress (`state=2`)** — pushing `state=3`
-  from any other state (Assigned, or already Resolved/Closed/Canceled) is silently rejected (HTTP
-  200, empty body — see the empty-body gotcha). `hold_reason=10` is not a valid choice on this
-  instance and is stored empty/dropped, but native On Hold still sticks without it.
+### Status fields (ServiceNow side) — CORRECTED 2026-08-20 (VENDOR-STATED, TRUE-81044)
+> **This is the third answer to "which field is authoritative" and the first one that is not an
+> inference.** TRUE-79922 concluded native `state` only; the 2026-07-20 pass reversed that to
+> `u_status_task`; TRUE-81044 re-reverses it. Both earlier conclusions were **inferred from
+> observed rejections**. This one is **stated directly by ServiceNow**, so prefer it.
+
+**ServiceNow's stated rule:** to put a DOE Incident On Hold, the mandatory attribute set is
+**`state` + `hold_reason` + `u_eta`, submitted TOGETHER in one payload.** A ServiceNow-side
+**business rule derives the client-facing `u_status_task` FROM `state`** — so we must *not* write
+`u_status_task` ourselves. Doing so was fighting their derivation rule.
+
+- The native **`state`** field is what we drive. It is the SLA-bearing state machine *and*, via
+  their business rule, the source of the client-facing status. Push it with its two companions.
+  - ⚠ **SUPERSEDED:** the claim that ServiceNow "only permits the native On Hold transition FROM
+    In Progress (`state=2`)" was **inferred from empty-body 200 rejections and is now doubtful** —
+    an *incomplete mandatory attribute set* produces the identical silent rejection, and our
+    payload was in fact incomplete (see the empty-body gotcha). **The In-Progress gate has been
+    removed** (TRUE-81044). ServiceNow explicitly invited us to report back if holds still fail
+    from `Assigned`.
+- **`hold_reason`** is mandatory. ⚠ **SUPERSEDED:** the claim that `hold_reason=10` "is not a valid
+  choice on this instance and is stored empty/dropped" was **WRONG** — code **10 ("Other") IS
+  valid**, but it additionally **requires the companion free-text field
+  `u_other_reason_on_hold`**. We were sending `10` with no companion — an incomplete mandatory set
+  — which is why it read as dropped.
+- **`u_other_reason_on_hold`** (string) — required whenever `hold_reason=10`. Must never be empty.
+- **`u_eta`** is mandatory on every hold push, **unconditionally** — not "only when it differs from
+  the remote value" (the old conditional silently dropped a mandatory attribute on any re-push with
+  an unchanged or null ETA).
+- **`u_status_task` (custom) is the CLIENT-FACING status, but is now READ-ONLY to us** — DOE's own
+  ServiceNow views/reporting read it (`Open`, `In Progress`, `On Hold`, …) and their business rule
+  populates it from `state`. ⚠ **SUPERSEDED:** the 2026-07-20 instruction that "a hold MUST be
+  written here or the client never sees it" no longer holds — we removed both the hold and the
+  release writes. It remains the field **three code paths still READ** for hold detection (see the
+  gotcha on that outstanding dependency).
 - Numeric `state` codes seen on the wire: INC `On Hold`=**3**, `In Progress`=**2**;
-  RITM `On Hold`=**8**, the scheduled-update state=**"-14"**.
+  RITM `On Hold`=**8**, the scheduled-update state=**"-14"**. `hold_reason` `Other`=**10**.
 
-### Outbound (TOGaDesk → ServiceNow) — INC hold push (worker #1676 + #1677)
-When a DOE repair order is on hold, `send_ticket_updates.php` pushes:
-- **Always set `u_status_task='On Hold'`** — the client-facing field, writable from any active
-  state; on release set it back to `'In Progress'`. This is what actually surfaces the hold to the
-  client. Until the local status stops reverting (the library `qqStatus()` fix below) the outbound
-  never sees `orderIsOnHold=true` and never pushes `u_status_task` at all — hence the deploy
-  dependency.
-- **Set native `state=3` (On Hold) only when the incident is currently `In Progress`** — an SLA
-  pause. Because ServiceNow rejects the transition from any other state (empty-body 200), the cron
-  **defers without advancing `dtSynced`** when the incident is neither On Hold nor In Progress, and
-  re-pushes only once it reaches In Progress.
-- **Closed-incident reconcile:** if the incident is already `Resolved`/`Closed`/`Canceled` on the
+### Outbound (TOGaDesk → ServiceNow) — INC hold push (TRUE-81044 current shape)
+When a DOE repair order is on hold, `send_ticket_updates.php` pushes **the full mandatory set
+together, unconditionally**:
+- **`state=3` + `hold_reason=10` + `u_eta` + `u_other_reason_on_hold` in ONE payload, every run.**
+  No gate on the remote state, and no "only if the ETA changed" condition — either would drop a
+  mandatory attribute and earn a silent empty-body 200.
+- **`u_other_reason_on_hold`** is populated from the analyst's hold note already being read from
+  `repair_order_notes` for `work_notes`, falling back to
+  `App_Model_TogaDesk_RepairOrder::statusDisplay($repairOrder->status)` so it is **never empty**.
+- **`hold_reason` stays pinned to `SN_HOLD_REASON_DEFAULT` (10, "Other").** See the *decision* on
+  not mapping hold reasons, below.
+- **No `u_status_task` write** on either the hold push or the hold release — ServiceNow's own
+  business rule derives it from `state`. The `SN_TASK_STATUS_ON_HOLD` / `SN_TASK_STATUS_IN_PROGRESS`
+  constants and the `$snTaskStatus` read were removed with it.
+- **Idempotency + release both key off `state`, not `u_status_task`:** the "already held" guard is
+  `$snState != 'On Hold'` and the release trigger is `$snState == 'On Hold'`. **Rule: gate a push on
+  the field you actually write** — the old `$snTaskStatus` keys gated the push on a field we no
+  longer drive.
+- **Closed-incident reconcile (unchanged, still correct):** if the incident is already
+  `Resolved`/`Closed`/`Canceled` on the
   SNOW side it can no longer be held — the cron **advances `dtSynced`** to accept the remote
   closure instead of retrying forever. (A canceled incident, INC2190846, had generated **2,392**
   failed empty-body PATCHes before this.)
 - The "Assigned → In Progress" auto-start block is still guarded so it never pushes `state:2` over
   a held order (TRUE-79922). Named constants: `SN_STATE_IN_PROGRESS` / `SN_STATE_ON_HOLD` /
-  `SN_HOLD_REASON_DEFAULT` in `send_ticket_updates.php`, plus `SN_TASK_STATUS_ON_HOLD` /
-  `SN_TASK_STATUS_IN_PROGRESS` for the `u_status_task` writes (#1677).
-- `send_request_item_updates.php` (RITM) pushes `state:"8"` (On Hold) and carries a defensive
-  `HOLD_*` guard on the scheduled-update path so it can't push `state:"-14"` while held; its
-  reverse-hold logic keys off `u_status_task == 'On Hold'`.
+  `SN_HOLD_REASON_DEFAULT` in `send_ticket_updates.php`.
+- The existing **empty-body patch-response check** (~L493) still **throws**, so any remaining
+  rejection surfaces loudly instead of silently. Keep it.
+
+> **DECISION (TRUE-81044) — do NOT map TogaDesk hold reasons onto ServiceNow hold-reason codes.**
+> `hold_reason` stays pinned to `10` ("Other") and the human-readable reason is carried entirely in
+> `u_other_reason_on_hold` / `work_notes`, sourced from TogaDesk. Rationale: ServiceNow's own states
+> are not reliable, so mapping our six `HOLD_*` constants onto their code set buys nothing and adds
+> a second source of truth. **Do not re-propose a reason-code mapping in a future session.**
+
+### RITM cron — deliberately unchanged by TRUE-81044
+`send_request_item_updates.php` already pushes native `state:"8"` (On Hold) and **never wrote**
+`u_status_task`, so TRUE-81044's ask was already satisfied there. It carries a defensive `HOLD_*`
+guard on the scheduled-update path so it can't push `state:"-14"` while held; its reverse-hold logic
+still **reads** `u_status_task == 'On Hold'`.
+> **OPEN QUESTION:** ServiceNow's communication was scoped to **Incident** state. We did **not**
+> extend `hold_reason` / `u_eta` / `u_other_reason_on_hold` to the RITM payload without
+> confirmation. Whether the RITM On Hold transition carries the same mandatory-set requirement is
+> unconfirmed — ask before changing the RITM payload.
 
 ### Inbound (ServiceNow → TOGaDesk)
 - `process_tickets.php` reads the client-facing **`u_status_task`** field
@@ -125,6 +164,8 @@ When a DOE repair order is on hold, `send_ticket_updates.php` pushes:
   bare **`STATUS_HOLD`** on the local repair order (this is exactly why the library `qqStatus()`
   fix below had to recognize bare `HOLD`). Both inbound "Assigned-force" `patchINC(state:2)` blocks
   are guarded against a local hold.
+- **Since TRUE-81044 this read is a DEPENDENCY on ServiceNow's own business rule**, not on anything
+  we write — the outbound no longer sets `u_status_task`. See the outstanding-dependency gotcha.
 
 ### RITM "ETA-in-hold"
 - `send_request_item_updates.php` syncs `scheduled_date` (ETA) to ServiceNow **inside the
@@ -208,19 +249,32 @@ The SNOW round-trip note reconciliation (delete-and-reinsert in `process_tickets
   auto-start fought the hold push — `state:3` then `state:2` pushed seconds apart, every run,
   so SNOW never stayed On Hold. Any future auto-start/auto-status block on these crons MUST
   short-circuit when the local RO is in a `HOLD_*` status.
-- **⚠ SUPERSEDED (see Change history 2026-07-20):** TRUE-79922 originally concluded that reading
-  `u_status_task` (instead of native `state`) was a bug and that hold detection should be on native
-  `state` only. That was **backwards.** `u_status_task` is the client-facing authoritative field
-  and **is** the correct hold-detection field — the live inbound cron reads it at
-  `process_tickets.php:206`. Native `state` is the internal SLA state, pushed only as a secondary
-  pause when the incident is In Progress.
-- **The empty-body 200 (nycd3 `data/tasks` PATCH):** `App_Api_NYCDOEV2::patchINC`/`patchRITM`
-  (both hit `/api/nycd3/data/tasks/{number}`) return **HTTP 200 with an EMPTY body (no `result`
-  object)** when a requested native `state` transition is illegal — e.g. pushing `state=3` from a
-  non-In-Progress state. This is silent: no exception, no error field. A canceled incident
-  (INC2190846) generated **2,392** such failed PATCHes before the closed-incident reconcile was
-  added. Gate native `state=3` on `snState == 'In Progress'`, and reconcile
-  Resolved/Closed/Canceled by advancing `dtSynced`.
+- **⚠ The "which field is authoritative" question has flipped TWICE. Read this before changing it a
+  fourth time.** TRUE-79922 said native `state` only (inferred). 2026-07-20 reversed it to
+  `u_status_task` (inferred from test PATCHes). **TRUE-81044 re-reversed it on ServiceNow's own
+  written instruction:** we write `state` + `hold_reason` + `u_eta` (+ `u_other_reason_on_hold`) and
+  **never write `u_status_task`**, because their business rule derives it from `state`. Only the
+  current answer is vendor-stated; do not re-litigate it from observed behavior alone.
+- **⚠ An empty-body 200 does NOT tell you WHY (nycd3 `data/tasks` PATCH).**
+  `App_Api_NYCDOEV2::patchINC`/`patchRITM` (both hit `/api/nycd3/data/tasks/{number}`) return
+  **HTTP 200 with an EMPTY body (no `result` object)** on rejection — no exception, no error field.
+  **Critically, this response is IDENTICAL for an illegal state transition and for an INCOMPLETE
+  mandatory attribute set.** The previously documented rule "On Hold is only permitted FROM In
+  Progress" was *inferred* from this response and is now doubtful: our payload was genuinely
+  incomplete in two ways (`u_eta` sent only when it differed from the remote value, so any re-push
+  with an unchanged or null ETA dropped a mandatory attribute; and `u_other_reason_on_hold` never
+  sent at all alongside `hold_reason=10`). **Diagnostic rule for this integration: never infer a
+  state-machine rule from an empty-body 200 until you have verified payload completeness first.**
+  The In-Progress gate was removed on that basis. Separately, a canceled incident (INC2190846)
+  generated **2,392** such failed PATCHes before the closed-incident `dtSynced` reconcile was added.
+- **⚠ OUTSTANDING DEPENDENCY (open, NOT closed by TRUE-81044) — we stopped WRITING `u_status_task`
+  but three paths still READ it.** Hold detection and RITM hold release now depend on ServiceNow's
+  business rule actually populating that field from `state`. If it doesn't, these break **silently**:
+  - `worker/crons/sync/nycdoe/process_tickets.php:206` — inbound hold detection
+  - `worker/crons/sync/nycdoe/send_request_item_updates.php:224` — RITM reverse-hold
+  - `worker/crons/sync/nycdoe/send_request_item_updates.php:139` — RITM closed-state check
+  **Verification owed on the FIRST hold that syncs after deploy:** read the incident back and confirm
+  `u_status_task` returns `On Hold`, via legacy `Logs.API` filtered `endpoint LIKE '%nycd3%'`.
 - Holds are **never auto-released** — if a SNOW user manually changes status, the outbound
   cron correctly re-asserts On Hold; that is intended, not a bug. Clear the hold in TOGaDesk.
 - **Diagnosing which writer dropped a hold:** `repair_order_history`
@@ -253,6 +307,26 @@ The SNOW round-trip note reconciliation (delete-and-reinsert in `process_tickets
   `repair_orders.status` directly cannot distinguish the six hold variants after a recompute.
 
 ## Change history
+- 2026-08-20 — **TRUE-81044: re-corrected the field-authority model on ServiceNow's own written
+  instruction, and fixed the real root cause of the failing INC hold push.** ServiceNow stated the
+  mandatory attribute set for an Incident On Hold is **`state` + `hold_reason` + `u_eta` together**,
+  and that their business rule derives the client-facing `u_status_task` **from `state`** — so our
+  `u_status_task` writes were fighting it. `send_ticket_updates.php`: removed `u_status_task` from
+  both the hold push and the release (and the `SN_TASK_STATUS_*` constants / `$snTaskStatus` read);
+  now sends `state`+`hold_reason`+`u_eta` **unconditionally together** (the `In Progress` gate and
+  the "only if ETA differs" condition are **gone**); added `u_other_reason_on_hold`, required
+  alongside `hold_reason=10` ("Other" IS valid — the earlier "invalid/dropped" note was wrong) and
+  sourced from the analyst hold note with a `statusDisplay()` fallback so it is never empty;
+  re-keyed the idempotency guard and release trigger from `$snTaskStatus` to `$snState`. Root-cause
+  correction: the documented "On Hold only from In Progress" rule was **inferred** from empty-body
+  200s, which do not distinguish an illegal transition from an **incomplete mandatory set** — ours
+  was incomplete. Decision: **no** mapping of the six TogaDesk `HOLD_*` constants onto ServiceNow
+  hold-reason codes; reason text comes from TogaDesk only. RITM cron intentionally untouched (already
+  compliant); open question whether RITM needs the same mandatory set. New open item: three paths
+  still **read** `u_status_task` — verify it comes back `On Hold` on the first post-deploy hold.
+  worker-only, single file (commit `ead92ad1e`, branch `TRUE-81044`, 34/-43); `php -l` clean on PHP
+  7.2, so the `library`-before-`togadesk` deploy-ordering constraint does not apply here.
+  (mhammontree)
 - 2026-07-20 — **Corrected the authoritative-field model and shipped the INC hold push.**
   Controlled test PATCHes proved the earlier "native `state` is authoritative; never use
   `u_status_task`" conclusion (TRUE-79922) **backwards**: the custom **`u_status_task`** is the

package/package.json

@@ -1,6 +1,6 @@
 {
   "name": "toga-ai",
-  "version": "1.0.625",
+  "version": "1.0.626",
   "description": "TOGA Technology Team Claude Knowledge System — shared AI coding harness with skills, knowledge base CLI, and project installer for Claude Code.",
   "keywords": [
     "claude",