toga-ai 1.0.770 → 1.0.772

package/knowledge/sessions/2026-09-03-commerce-query-cache-diagnosis-apeterson.md

@@ -0,0 +1,225 @@
+---
+type: session
+slug: commerce-query-cache-diagnosis
+title: Root-caused toga2-commerce stale-cache + stuck-login bugs against toga25-supply's fixed setup
+author: apeterson
+repos: [toga25-supply, toga2-commerce]
+framework: "2.0"
+client: shared
+status: active
+created: 2026-09-03
+updated: 2026-09-03
+---
+
+# Session: commerce-query-cache-diagnosis
+**Date:** 2026-09-03
+**Project/Repo:** toga2-commerce + toga25-supply (2.0)
+**Task:** Understand what `supply-chain-query-cache` does in toga25-supply, then use it as the reference to root-cause toga2-commerce's chronic stale-data, failed-cache-clear, and stuck-login-page bugs.
+
+**No code was written this session.** It is a read-only investigation. Everything below is a
+verified finding plus an agreed fix plan that has NOT been implemented.
+
+---
+
+## What WORKED
+
+**Mapped `supply-chain-query-cache` in toga25-supply completely (verified in source):**
+- It is not a hand-written cache. It is the whole React Query cache dumped to `localStorage`
+  under that key, via `createSyncStoragePersister` at `src/App.tsx:27-30`, wired by
+  `PersistQueryClientProvider` at `src/App.tsx:33`.
+- Persists **every successful query** except three namespaces, excluded in
+  `shouldDehydrateQuery` (`src/App.tsx:63-70`): `talos`, `surface-meta`, `surface-meta-group`.
+  Surface-meta is excluded because it carries permission/feature-gate state, so a persisted
+  copy would hide a grant or a revoke.
+- QueryClient default is `staleTime: 0` (`src/App.tsx:18`) precisely because the cache is
+  persisted.
+- Resets on: login (`contexts/AuthContext.tsx:113-114`), logout (`:141-146`), each list page's
+  Refresh button (`queryClient.clear()` + `resetQueries(["table-data"])` — SalesOrders,
+  TransferOrders, ServiceRequests, Items, VendorItems, Bundles, Inventory), and a 24h
+  `maxAge`.
+- `maxAge` default confirmed at `node_modules/@tanstack/query-persist-client-core/build/modern/persist.js:10`
+  — one timestamp for the **whole blob**, not per query.
+- No `buster` is set, so a deploy does not invalidate it. `DeploymentGuard`
+  (`src/hooks/useDeploymentGuard.tsx`) handles deploys separately by watching the Core
+  parameter `META_LAST_REFRESH_DATETIME` and signing the user out.
+- Query-key → endpoint map: `["table-data", slug, …]` → `getDataTableData`;
+  `["table-meta", slug]` → `GET /table-views/meta` (blox
+  `components/Table/hooks/useFetchTablePageMeta.js:8`); `["status-colors", route]` →
+  `GET /{route}?fields=slug,colorHex&sort=sortOrder`; current user → `GET /users`;
+  `["deploymentVersion"]` → `GET /core-parameters`.
+- Remaining non-zero `staleTime` hooks in supply that ARE persisted (so each can serve a
+  previous session's data and never fetch): `useClientFields`, `useCurrentUser`,
+  `useStatusTypeValues`, `useStatusColors`, `useSalesOrderUuidForServiceRequest` (all
+  `Infinity`); the three VendorItem form hooks (`5 * 60_000`); `useTalosThreads` (`30_000`).
+  `useFetchSurfaceMeta`'s `Infinity` is safe only because it is excluded from persisting.
+
+**Root-caused four stacked bugs in toga2-commerce (branch `TRUE-81610`), all verified:**
+
+1. **App-wide 24-hour `staleTime`** — `src/App.tsx:26-31`. Supply's default is `0`. Combined
+   with a persisted cache (key `"commerce"`, `src/App.tsx:34-37`) and **no**
+   `dehydrateOptions` (nothing excluded), yesterday's queries are served today with no
+   network call. This is the "users not seeing updated data" report, and it happens daily —
+   not only after a deploy. The same 24h value is also hardcoded per-hook at
+   `pages/BundleView/hooks/useBundleQueryData.ts:26,40` and
+   `pages/Home/viewModel/useHomeViewModel.ts:150,173,197`, so changing only the default is
+   not enough.
+
+2. **Cache clears are not gated on restoration** — `PersistQueryClientProvider` renders
+   children *during* restore (confirmed in
+   `node_modules/@tanstack/react-query-persist-client/build/modern/PersistQueryClientProvider.js`:
+   `isRestoring` starts `true`, `restoreClient` resolves later). A `clear()` +
+   `removeItem("commerce")` that runs before restore finishes is silently undone when the
+   pending restore hydrates the old cache on top and the subscription writes it back.
+   Unguarded in all three places: `contexts/AuthContext.tsx:180-183`,
+   `components/NavIcons/NavIconList.tsx:162-170`, `api/axiosInstance.ts:46`. Supply fixed
+   this with an `awaitRestoration()` promise at `contexts/AuthContext.tsx:56-77`. **This is
+   the "it's not successfully clearing their caches" report.**
+
+3. **Stuck/broken login page — two gates check different keys.** The axios request
+   interceptor gates on **`user`** (`api/axiosInstance.ts:112-118`): with no `user` it calls
+   `fetchPublicToken()`, which POSTs `/auth/public` and **writes the public token to
+   `localStorage` as `accessToken`** (`:53-64`). But `hooks/useAuthenticationFlow.ts:213-217`
+   gates on **`accessToken`** and returns early if it exists. So after a forced logout: the
+   first API call writes a public `accessToken` back, the auth flow sees it, skips both the
+   SSO redirect and `/login`, and the user is logged out while the app thinks they are not.
+   Supply already gates on `user` — see the explicit comment at
+   `contexts/AuthContext.tsx:47-51`.
+
+4. **The forced logout is too aggressive.** `contexts/AuthContext.tsx:192-252` watches
+   `META_LAST_REFRESH_DATETIME` and on any change calls `logout()` +
+   `window.location.href = "/"` with no notice. It also **monkey-patches
+   `window.history.pushState` and `replaceState`** (`:216-226`) to refetch on every
+   navigation — react-router uses those same methods. On an SSO client, `/` re-triggers the
+   IdP. Supply's DeploymentGuard at least shows a 2.5s toast first.
+   Separately, commerce's `login()` (`:100-148`) **never clears the cache**, so signing in as
+   a different user inherits the previous user's data for up to 24h. Supply clears on login.
+
+**Resolved the "but caching is what makes items/bundles fast" objection — `staleTime` is the
+wrong knob:**
+- React Query paints cached data instantly regardless of `staleTime`. `staleTime` only
+  controls whether it *also* refetches in the background. `gcTime` is the knob that keeps the
+  cached copy around.
+- Verified commerce gates **every** loading state on `isLoading`, never `isFetching`
+  (`useBundleQueryData.ts:22,35`; `useHomeViewModel.ts:136,157,177`). `isLoading` is false
+  whenever cached data exists. So dropping `staleTime` shows **no** spinner on cached data —
+  the 24h `staleTime` buys nothing on perceived speed, it only suppresses the refetch.
+- **`gcTime` is set nowhere in commerce**, so it is the 5-minute default. Navigating away
+  from Home for 6+ minutes and back drops the query from memory and *does* show a real
+  spinner today, despite the 24h staleTime. The persisted cache only helps on a full reload,
+  not in-session navigation. Fixing `gcTime` is the actual speed win.
+
+---
+
+## What did NOT work — DO NOT RETRY THESE
+
+- **Nothing was attempted and failed** — no code was written or run this session.
+
+Two dead ends established by reasoning, worth not re-proposing:
+
+- **A buster alone will not fix commerce.** `buster` is compared once on restore and deletes
+  the persisted cache when it differs (`persist.js:20-24`). It only fires on a **deploy**.
+  Commerce's stale data is a daily 24h-`staleTime` problem, so a buster leaves the main bug
+  untouched. It is step 5 of 5, not step 1.
+- **Versioning the storage key instead of using `buster`** (e.g.
+  `supply-chain-query-cache-v2`) works but leaks the old blob in `localStorage` forever.
+  `buster` calls `removeClient()` and actually cleans up. Use `buster`.
+- **Do not expect a buster to fix a cached JS bundle.** Buster runs *inside* the app; if the
+  browser never downloaded the new app, buster never runs. Diagnostic to separate the two:
+  hard-refresh (Cmd+Shift+R) fixes it → bundle/CDN cache-header problem, buster is
+  irrelevant; only logout fixes it → persisted query data, buster helps.
+
+---
+
+## Not tried yet (candidates for next session)
+
+1. Set `gcTime: 1000 * 60 * 60` + `staleTime: 30_000` + `refetchOnWindowFocus: false` on the
+   heavy commerce hooks (`pages/BundleView/hooks/useBundleQueryData.ts`,
+   `pages/Home/viewModel/useHomeViewModel.ts`). Keeps the instant paint, makes data fresh.
+2. Change the commerce app default `staleTime` at `src/App.tsx:29` from
+   `1000 * 60 * 60 * 24` to `0`.
+3. Port supply's `awaitRestoration()` gate (`toga25-supply/src/contexts/AuthContext.tsx:56-77`)
+   into commerce and put all three clear sites behind it.
+4. Change `hooks/useAuthenticationFlow.ts:213` to gate on `localStorage.getItem("user")`
+   instead of `accessToken`.
+5. Add a `clear()` to commerce's `login()`, gated on restoration.
+6. Add the buster to **both** apps: a `define: { __BUILD_ID__: JSON.stringify(process.env.AWS_COMMIT_ID ?? "dev") }`
+   block in `vite.config.ts`, a `declare const __BUILD_ID__: string;` in `src/vite-env.d.ts`,
+   and `buster: __BUILD_ID__` in `persistOptions`. Note `package.json` version is `0.0.0` in
+   both repos (never bumped, unusable as a build id) and there is **no `amplify.yml` in
+   either repo** — Amplify is console-configured, so `AWS_COMMIT_ID` is the available value.
+7. Check whether prices or stock counts ride in the bundle/item payloads. If they do, those
+   specific queries must gate their spinner on `isFetching`, not `isLoading` — a wrong price
+   shown for one second is a real problem, unlike a stale item list.
+8. Check the Amplify/CloudFront cache headers on `index.html` for commerce — the second,
+   separate cache that a buster cannot touch.
+9. Audit whether the commerce item/bundle requests are field-limited and what FK depth they
+   use (api2 default depth is 3). Caching is currently hiding an oversized payload.
+10. Revisit supply's own persisted `staleTime: Infinity` hooks (`useClientFields`,
+    `useCurrentUser`, `useStatusTypeValues`, `useStatusColors`,
+    `useSalesOrderUuidForServiceRequest`) — each can serve a previous session's data with
+    nothing in the network tab.
+
+---
+
+## Current file state
+
+| File | Status | Notes |
+|------|--------|-------|
+| (none) | unchanged | Read-only investigation. No file in any repo was created, modified, or deleted. |
+
+Working tree note: `toga2-commerce` was inspected on branch `TRUE-81610`; `api2` (the session's
+primary dir) is on `_sandbox-client` with only the pre-existing `CLAUDE.md` / `.claude/` /
+`.gstack/` changes from before this session.
+
+---
+
+## Decisions made
+
+- **`gcTime`, not `staleTime`, is the speed knob — fix both together.** Rationale: React Query
+  serves cached data immediately no matter what `staleTime` says, and commerce gates every
+  spinner on `isLoading` (false when cached data exists), so lowering `staleTime` costs
+  nothing visually. Raising `gcTime` is what actually removes the in-session spinner.
+  Rejected: keeping the 24h `staleTime` "for speed" — verified it buys no perceived speed and
+  is the direct cause of the stale-data reports.
+- **Fix order is staleTime/gcTime → restoration gate → auth gate → buster.** Rationale: ranked
+  by how often each bug bites. The 24h `staleTime` fires daily, the restoration race is why
+  every attempted clear fails, the `accessToken`/`user` mismatch is the stuck login page, and
+  the buster only matters on deploy day. Rejected: leading with the buster (which is what the
+  conversation started from) — it would have left the daily bug in place.
+- **Use `staleTime: 30_000` rather than `0` on the heavy commerce hooks.** Rationale: stops a
+  refetch storm when a user clicks Home → bundle → Home in a few seconds, while still never
+  surviving a session or a deploy. Rejected: `0` everywhere (too many repeats of a large
+  payload) and any value long enough to outlive a session.
+- **toga25-supply is the reference implementation; port from it rather than inventing.**
+  Rationale: supply already hit and fixed the same three traps (restoration race, public-token
+  gate, persisted-cache staleTime) and the fixes carry explanatory comments. Rejected: writing
+  fresh solutions in commerce.
+
+---
+
+## Blockers
+
+- **Unanswered question that changes step 7:** do the commerce bundle/item payloads carry
+  prices or stock/availability counts? If yes, those queries need an `isFetching`-based
+  loading state, not just the `gcTime`/`staleTime` swap. Asked the developer; not yet answered.
+- **No implementation authorization yet.** The plan was presented and the developer ran
+  `/session-save` instead of approving. Do not start writing code next session without
+  asking again.
+- Not a blocker, but note: `toga2-commerce` is on branch `TRUE-81610` with uncommitted work
+  (per the standards doc, commerce's blox pin and `dedupe` were being changed on this branch).
+  Check the tree state before editing.
+
+---
+
+## Exact next step
+
+> Ask the developer two things before writing any code: (1) do the commerce bundle/item
+> payloads include prices or stock counts, and (2) confirm authorization to implement. Then
+> start with candidate #1 — edit `toga2-commerce/src/pages/BundleView/hooks/useBundleQueryData.ts`
+> lines 26 and 40, replacing `staleTime: 1000 * 60 * 60 * 24` with
+> `staleTime: 30_000, gcTime: 1000 * 60 * 60, refetchOnWindowFocus: false`, and make the same
+> change at `src/pages/Home/viewModel/useHomeViewModel.ts:150,173,197`. Verify the instant
+> paint is unchanged before touching anything auth-related.
+
+---
+_Saved by /session-save on 2026-09-03_

package/package.json

@@ -1,6 +1,6 @@
 {
   "name": "toga-ai",
-  "version": "1.0.770",
+  "version": "1.0.772",
   "description": "TOGA Technology Team Claude Knowledge System — shared AI coding harness with skills, knowledge base CLI, and project installer for Claude Code.",
   "keywords": [
     "claude",

package/scripts/install.js

@@ -275,6 +275,130 @@ function countMd(dir) {
   return n;
 }
 
+/* ------------------------------------------------------------------ */
+/* User-level telemetry env (~/.claude/settings.json)                   */
+/* ------------------------------------------------------------------ */
+
+// Canonical OTEL block owned by the harness. These keys are MANAGED: on every
+// install they are removed and re-written from this object, so a stale value
+// can never survive an upgrade. Every other key in the user's settings.json —
+// including env vars the developer added themselves — is left exactly as-is.
+const TELEMETRY_ENV = {
+  CLAUDE_CODE_ENABLE_TELEMETRY: '1',
+  CLAUDE_CODE_ENHANCED_TELEMETRY_BETA: '1',
+  CLAUDE_CODE_OTEL_CONTENT_MAX_LENGTH: '61440',
+  OTEL_EXPORTER_OTLP_ENDPOINT: 'https://otel.togaiq.com',
+  OTEL_EXPORTER_OTLP_HEADERS: "Authorization=Bearer a7b2f7030950196206483141539865a944cec91465877eadecb389ca3323090f",
+  OTEL_EXPORTER_OTLP_PROTOCOL: 'http/protobuf',
+  OTEL_LOGS_EXPORTER: 'otlp',
+  OTEL_LOGS_EXPORT_INTERVAL: '5000',
+  OTEL_LOG_ASSISTANT_RESPONSES: '1',
+  OTEL_LOG_TOOL_DETAILS: '1',
+  OTEL_LOG_USER_PROMPTS: '1',
+  OTEL_METRICS_EXPORTER: 'otlp',
+  OTEL_METRICS_INCLUDE_ACCOUNT_UUID: 'true',
+  OTEL_METRICS_INCLUDE_ENTRYPOINT: 'true',
+  OTEL_METRICS_INCLUDE_RESOURCE_ATTRIBUTES: 'true',
+  OTEL_METRICS_INCLUDE_SESSION_ID: 'true',
+  OTEL_METRICS_INCLUDE_VERSION: 'true',
+  OTEL_METRIC_EXPORT_INTERVAL: '60000',
+  OTEL_TRACES_EXPORTER: 'otlp',
+  OTEL_TRACES_EXPORT_INTERVAL: '5000',
+};
+
+// Per-machine endpoint/token override, so a developer can point at a different
+// collector without editing the bundle.
+function telemetryEnvBlock() {
+  const block = Object.assign({}, TELEMETRY_ENV);
+  if (process.env.TOGA_OTEL_ENDPOINT) {
+    block.OTEL_EXPORTER_OTLP_ENDPOINT = process.env.TOGA_OTEL_ENDPOINT;
+  }
+  if (process.env.TOGA_OTEL_TOKEN) {
+    block.OTEL_EXPORTER_OTLP_HEADERS = 'Authorization=Bearer ' + process.env.TOGA_OTEL_TOKEN;
+  }
+  return block;
+}
+
+// Writes the managed OTEL keys into the USER settings file (~/.claude/settings.json).
+// Surgical: parses the existing JSON, replaces only the managed keys under `env`,
+// and rewrites the file. Never rewrites a file it could not parse.
+// Opt out with TOGA_SKIP_TELEMETRY=1.
+function applyTelemetryEnv() {
+  if (process.env.TOGA_SKIP_TELEMETRY === '1') {
+    return { status: 'skipped', reason: 'TOGA_SKIP_TELEMETRY=1' };
+  }
+
+  const userClaudeDir = path.join(os.homedir(), '.claude');
+  const settingsPath = path.join(userClaudeDir, 'settings.json');
+
+  let settings = {};
+  const exists = fs.existsSync(settingsPath);
+  if (exists) {
+    let raw;
+    try {
+      raw = fs.readFileSync(settingsPath, 'utf8');
+    } catch (e) {
+      return { status: 'error', reason: 'unreadable: ' + e.message };
+    }
+    if (raw.trim()) {
+      try {
+        settings = JSON.parse(raw);
+      } catch (e) {
+        // Invalid JSON — refuse to touch it rather than destroy the user's settings.
+        return { status: 'error', reason: 'existing settings.json is not valid JSON — left untouched' };
+      }
+      if (settings === null || typeof settings !== 'object' || Array.isArray(settings)) {
+        return { status: 'error', reason: 'existing settings.json is not a JSON object — left untouched' };
+      }
+    }
+  }
+
+  const block = telemetryEnvBlock();
+  const currentEnv = (settings.env && typeof settings.env === 'object' && !Array.isArray(settings.env))
+    ? settings.env
+    : {};
+
+  // Rebuild env: keep every non-managed key in its original order, then append
+  // the managed keys from the canonical block (delete-then-add semantics).
+  const nextEnv = {};
+  for (const key of Object.keys(currentEnv)) {
+    if (!(key in block)) nextEnv[key] = currentEnv[key];
+  }
+  Object.assign(nextEnv, block);
+
+  if (exists && JSON.stringify(currentEnv) === JSON.stringify(nextEnv)) {
+    return { status: 'unchanged', path: settingsPath };
+  }
+
+  settings.env = nextEnv;
+
+  try {
+    fs.mkdirSync(userClaudeDir, { recursive: true });
+    if (exists) fs.copyFileSync(settingsPath, settingsPath + '.toga-bak');
+    // Atomic write so an interrupted install can't leave a half-written file.
+    const tmp = settingsPath + '.toga-tmp';
+    fs.writeFileSync(tmp, JSON.stringify(settings, null, 2) + '\n');
+    fs.renameSync(tmp, settingsPath);
+  } catch (e) {
+    return { status: 'error', reason: e.message };
+  }
+
+  return { status: exists ? 'updated' : 'created', path: settingsPath };
+}
+
+function reportTelemetryEnv() {
+  const r = applyTelemetryEnv();
+  const n = Object.keys(TELEMETRY_ENV).length;
+  if (r.status === 'error') {
+    console.log('  ✗ Telemetry (~/.claude/settings.json: ' + r.reason + ')');
+  } else if (r.status === 'skipped') {
+    console.log('  · Telemetry (skipped — ' + r.reason + ')');
+  } else {
+    console.log('  ✓ Telemetry (' + n + ' OTEL env vars ' + r.status + ' in ~/.claude/settings.json)');
+  }
+  return r;
+}
+
 /* ------------------------------------------------------------------ */
 /* settings.json merge                                                  */
 /* ------------------------------------------------------------------ */
@@ -572,6 +696,11 @@ function main() {
   }
   console.log('');
 
+  // User-level OTEL env — applies on every install, including the bare
+  // `npm i -g toga-ai` postinstall that never reaches a project directory.
+  reportTelemetryEnv();
+  console.log('');
+
   if (isPostInstall && !repoFlag) {
     console.log('  First-time setup: run `toga-ai` from your project to');
     console.log('  install the harness into your .claude/ directory.');