@rubytech/create-sitedesk-code 0.1.508 → 0.1.509

package/payload/platform/docs/superpowers/specs/2026-07-26-task-1926-portal-push-concurrency-guard-design.md

@@ -0,0 +1,200 @@
+# Task 1926 — same-account portal push concurrency guard (design)
+
+**Date:** 2026-07-26
+**Task:** [`.tasks/1926-concurrent-same-account-portal-push-lets-the-pre-stage-clear-delete-the-live-tree.md`](../../../../.tasks/1926-concurrent-same-account-portal-push-lets-the-pre-stage-clear-delete-the-live-tree.md)
+**Touches:** `platform/plugins/cloudflare/bin/portal-index-push.mjs`, `platform/plugins/cloudflare/mcp/__tests__/portal-index-push.test.ts`
+
+## Problem
+
+`pushAccount` stages an account's file index at `next = currentGeneration + 1`, flips
+the `directory_state` pointer to `next`, then sweeps every other generation. Task 1923
+added a pre-stage `DELETE FROM directory WHERE accountId = ? AND generation = ?` bound
+to `next`, so a stage that died mid-batch could self-heal instead of colliding forever
+on `UNIQUE (accountId, relPath, generation)`.
+
+Two pushes of one account break that. The second writer reads `current = 5` before the
+first writer's flip, computes `next = 6`, and by the time its pre-stage DELETE runs the
+pointer already names generation 6. The DELETE removes the live tree. A client listing
+in the gap between that DELETE and the re-insert sees an empty portal.
+
+The overlap is reachable: the admin server's 60s loop calls `pushAllAccounts`, and a
+hand-run `node portal-index-push.mjs --account <id>` calls the same `pushAccount` from a
+separate OS process.
+
+## What success looks like
+
+Two concurrent pushes of one account never leave a reader seeing an empty or partial
+tree, and never delete rows at the pointer's generation.
+
+## Why the three candidate approaches in the task file were rejected
+
+**A per-account advisory lock.** The two racers are separate OS processes. A JavaScript
+mutex inside the admin server does not exist in the CLI process, so it guards nothing.
+A lock that does work across processes has to live in D1, which means new columns, a
+lease, and a lease expiry. A lease expiry reintroduces the wedge class Task 1923 just
+removed: a holder that dies wedges the account until the lease runs out, and a push
+slower than its lease keeps writing after another writer has taken over.
+
+**`next` from `MAX(generation)` alone.** This stops two writers sharing a staging
+generation, but the slower writer still flips the pointer down onto its own generation
+after the faster writer has already swept it. The reader still sees an empty tree, so
+the success criterion is not met.
+
+**Scoping the clear to `generation = next AND generation != current`.** `current` is a
+value read at the top of the push. By the time the DELETE runs, the pointer has moved.
+Re-reading the pointer immediately before the DELETE narrows the window without closing
+it, because the peer can flip between the re-read and the DELETE.
+
+## Design
+
+Three changes to `pushAccount`. No lock, no lease, no schema change, no new table.
+
+### 1. Claim a generation above everything that exists
+
+Keep the existing pointer read, and add one statement beside it:
+
+```sql
+SELECT MAX(generation) AS g FROM directory WHERE accountId = ?
+```
+
+`next = Math.max(current, maxStagedGeneration) + 1`.
+
+A peer that has already inserted at least one row is visible in that MAX, so the two
+writers claim different generations and neither stages into the other's rows.
+
+Two statements rather than one nested `UNION ALL`: each is trivially readable, each is
+trivially fakeable in a test, and the pointer read stays the push's first statement,
+which an existing test asserts.
+
+### 2. The flip only ever moves the pointer forward
+
+```sql
+INSERT INTO directory_state (accountId, currentGeneration) VALUES (?, ?)
+ON CONFLICT (accountId) DO UPDATE SET currentGeneration = excluded.currentGeneration
+WHERE excluded.currentGeneration > directory_state.currentGeneration
+```
+
+SQLite has supported a `WHERE` clause on upsert's `DO UPDATE` since 3.24, and D1 is
+SQLite. This is what makes a losing writer harmless: it cannot drag readers back onto a
+generation the winner has already swept.
+
+The D1 client returns rows, not affected-row meta, so the flip is followed by a pointer
+readback. That readback is the branch point:
+
+- **Pointer equals `next`.** We won. Log `op=flip`, sweep, verify, return. Unchanged
+  behaviour.
+- **Pointer is above `next`.** A peer won. Log
+  `op=stage-contended account=<id> generation=<next> pointer=<actual> action=superseded`
+  and return without sweeping and without verifying. Both of those would measure the
+  peer's tree against our expected row count, and `op=verify` would read as a failure
+  when nothing failed.
+
+Our staged rows at the losing generation are left in place. They are invisible (no
+reader sees a generation the pointer does not name) and the winner's next sweep collects
+them.
+
+### 3. The sweep only ever removes older generations
+
+```sql
+DELETE FROM directory WHERE accountId = ? AND generation < ?
+```
+
+`< next` rather than `!= next`, so a writer never deletes a peer's higher in-flight
+stage. Rows above `next` from an abandoned stage still converge: the following cycle
+claims a generation above them and sweeps them once it flips.
+
+### 4. The Task 1923 pre-stage clear is removed
+
+Because `next` is now always above every existing generation for the account, the orphan
+`SELECT COUNT(*) ... generation = next` can only ever return 0 and the pre-stage DELETE
+can never match a row. Leaving unreachable code whose comment claims it is the wedge
+heal would be false narration, so the orphan count, the DELETE, the `op=stage-clear` log
+line and its two tests go.
+
+Task 1923's outcome is preserved by construction and is stronger than before. A stage
+that died mid-batch leaves rows at generation 6; the next run claims 7, so there is no
+UNIQUE collision to freeze on, and the gen-6 orphans are swept after the flip. The heal
+now happens with no DELETE before the flip at all, which is exactly what removes this
+task's hazard rather than narrowing it.
+
+This was confirmed with the operator before the spec was written, because Task 1926
+lists Task 1923's heal as out of scope.
+
+## Cases covered, and the one left fail-closed
+
+**Peer reads the generation after we have staged a row.** It claims a higher generation.
+Both pushes complete. The forward-only flip picks the winner. The reader sees the old
+complete tree, then one complete new tree. Neither push fails.
+
+**Both read the generation before either inserts.** Both claim the same generation. The
+second writer's first INSERT hits `UNIQUE (accountId, relPath, generation)` and that push
+throws. Both walks resolve the same exposed set and `walkExposed` emits the exposed dirs
+in the same order, so the second writer collides on its first row and contributes
+nothing. The reader is safe: the winner's tree is complete and the pointer never moved
+backward. The loser logs `op=failed` and the next 60s cycle succeeds.
+
+This second case is left fail-closed rather than retried. The task's success criterion is
+about what a reader can see, and a loud failure that self-corrects one cycle later meets
+it. A retry-on-collision path would be an invented requirement.
+
+**The one residual, stated rather than fixed.** The claim above relies on both walks
+starting with the same row. If the account's `SCHEMA.md` or `data-portal.json` changes in
+the window between the two generation reads, the two walks can begin with different rows,
+so the second writer contributes some non-overlapping rows at the shared generation before
+colliding on a shared one. The winner then flips onto its own complete tree plus those few
+extra rows. That is not an empty tree and not a partial one, so it does not breach the
+success criterion, and the next cycle's sweep removes the whole generation. No task file is
+filed for it because nothing is deferred: the case is inside what this design covers, and
+its outcome is a stale row for one cycle rather than a reader-visible break.
+
+## Observability
+
+`op=stage-contended account=<id> generation=<next> pointer=<actual> action=superseded`
+is emitted when a push's flip was superseded by a peer. This deviates from the task
+file's `action=<waited|skipped>` wording: the design never waits and never skips work, so
+neither value describes what happened. `superseded` names the actual outcome, and
+`pointer=` names the generation that won, which is what an operator needs to correlate
+the two runs.
+
+`op=stage-clear` is retired with the code that emitted it. The task file's line that a
+`stage-clear` with `cleared=` equal to the live tree size must never appear is satisfied
+absolutely: no `stage-clear` line can appear at all.
+
+## Testing
+
+Both new tests drive two interleaved `pushAccount` calls against one shared fake store
+and snapshot the reader's view after every statement. The assertion is the success
+criterion stated directly: the visible tree is never empty and never a strict subset of
+a complete tree.
+
+1. **The task file's interleave.** Writer B reads the generation, then pauses. Writer A
+   runs to completion. B resumes. Red on current code, because B's pre-stage DELETE binds
+   the generation the pointer now names and the snapshot goes empty. Green after, because
+   B collides on UNIQUE and A's tree is untouched. This test also asserts that no DELETE
+   is ever issued naming the pointer's current generation, which is the other half of the
+   success criterion.
+
+2. **The higher-claim interleave.** Writer A stages fully but pauses before its flip.
+   Writer B reads the generation, sees A's staged rows in the MAX, claims the next one up,
+   and completes. A resumes and its flip is a no-op. Asserts the pointer never moves
+   backward, that A emits `op=stage-contended` with `action=superseded`, and that A
+   issues no sweep.
+
+The existing 27 tests stay green apart from the two Task 1923 stage-clear tests, which
+are removed with the code they cover.
+
+## Scope boundaries
+
+**In scope:** the generation claim, the forward-only flip, the backward-only sweep, the
+removal of the pre-stage clear, and the two-writer regression tests.
+
+**Out of scope:** INSERT batching (Task 1842). Cross-account contention (separate
+generation counters, no contention). Any change to `runTargets`, `recordSuccess`, or the
+portal reader in `files.ts`.
+
+## Deployment
+
+`portal-index-push.mjs` ships inside the installer payload and is inlined into the admin
+server bundle. No D1 migration is required: every statement runs against the existing
+`directory` and `directory_state` tables as defined in
+`platform/plugins/cloudflare/skills/data-portal/schema.sql`.

package/payload/platform/plugins/admin/skills/platform-architecture/SKILL.md

@@ -1,7 +1,7 @@
 ---
 name: platform-architecture
 description: Use when grounding any documented-surface claim about what SiteDesk ships — plugins, skills, specialists, install/deploy flows, internals. This is the install catalogue, not evidence of what is enabled on the current account. For install state on this account, call `capabilities-here`; for documented surface, cite the `Source:` URL inline.
-content-hash: sha256:b15d0e491d957f75523ecf7056f8004bafcdf448767c0c3dd64b40a73926b58b
+content-hash: sha256:aabbb6dbe5d9f4fe8ff6f7b3e2f06a46bd1c5281ea46b820829ab2b3bc93a0c0
 brand: sitedesk-code
 product-name: SiteDesk
 ---
@@ -3189,8 +3189,8 @@ explicit `?plugin=` parameter, not inferred from `--` in the slug.
 
 | Route | Behaviour |
 |-------|-----------|
-| `GET /api/admin/agents` | Session-scoped. Lists the session account's public `agents/*/` dirs (never the `admin` agent), its user-created specialists from `plugins/*/agents/*.md`, and its shipped specialists, each row tagged `kind` and `origin` and carrying `risk`, `worstTool`, `unresolved` and `disabled`. Returns `{agents, accountId, skipped, specialistsSkipped, shippedSkipped}`. 401 when the session maps to no account. |
-| `GET /api/admin/agents/:slug` | Session-scoped. Without `?plugin=` or `?origin=`, returns a public agent's config fields + four owned docs + a `present` map (a missing or unreadable doc is `''`/`present.<role>=false`, never a 500). With `?plugin=`, that plugin's user-created specialist; with `?origin=shipped`, the shipped one. Both specialist shapes add `{risk, worstTool, unresolved, byTool, disabled}`, where `byTool` is one class per declared tool. Selection is explicit rather than inferred, because a premium file and a user-created specialist under a plugin of the same name produce the same slug. Unknown 404s, 401 as above. |
+| `GET /api/admin/agents` | Session-scoped. Lists the session account's public `agents/*/` dirs (never the `admin` agent), its user-created specialists from `plugins/*/agents/*.md`, and its shipped specialists, each row tagged `kind` and `origin` and carrying `risk`, `riskiestTool`, `unresolved` and `disabled`. Returns `{agents, accountId, skipped, specialistsSkipped, shippedSkipped}`. 401 when the session maps to no account. |
+| `GET /api/admin/agents/:slug` | Session-scoped. Without `?plugin=` or `?origin=`, returns a public agent's config fields + four owned docs + a `present` map (a missing or unreadable doc is `''`/`present.<role>=false`, never a 500). With `?plugin=`, that plugin's user-created specialist; with `?origin=shipped`, the shipped one. Both specialist shapes add `{risk, riskiestTool, unresolved, byTool, disabled}`, where `byTool` is one class per declared tool. Selection is explicit rather than inferred, because a premium file and a user-created specialist under a plugin of the same name produce the same slug. Unknown 404s, 401 as above. |
 | `POST /api/admin/agents/:slug/disable?accountId=` | Shipped agents only, enforced here: a slug present in none of the three shipped directories 404s. Moves the file from `specialists/agents/` to `specialists/agents-disabled/` and records the basename in the 0600 store. Returns `{ok, moved}`; `moved:false` means the agent existed only as a bundled template, so nothing was there to move and the store entry is what stops the next provisioning run delivering it. The store is read strictly before anything moves, so an unreadable store 500s with the file untouched rather than rewriting the file whole from an empty set and dropping every other disabled agent. Same `accountId` contract as delete. |
 | `POST /api/admin/agents/:slug/enable?accountId=` | Covers both of disable's outcomes. Returns `{ok, restored}`: a quarantined file moves back (`restored:true`); a bundled-only agent has nothing to move, so clearing the store entry is the whole job (`restored:false`), which is what stops provisioning withholding it. Treating that second case as "nothing to restore" made disable a one-way door. 404 only when neither directory nor store knows the agent. |
 | `DELETE /api/admin/agents/:slug?accountId=` | Public agents only, on the named validated account. Removes the dir after `deleteAgentProjection`; refuses the `admin` slug (403) and a missing/unknown `accountId` (400) with no write. Loud-fail: a graph-cleanup throw aborts the file removal. |
@@ -3220,10 +3220,15 @@ Agents page survived undiagnosed on a multi-account install.
   `reason=<graph-cleanup-failed|rm-error>` on failure (files preserved on a
   graph-cleanup throw).
 - `op=classify account=<id8> agent=<name> origin=<shipped|public|specialist>
-  risk=<green|amber|red> worstTool=<tool> tools=<n> unresolved=<comma-list|none>
+  risk=<green|amber|red> riskiestTool=<tool> tools=<n> unresolved=<comma-list|none>
   disabled=<bool>` — one line per classified agent per listing, so it fires on
-  every page load. `worstTool` makes a wrong class diagnosable without
-  re-deriving it. **A non-empty `unresolved=` is the one to act on**: the agent
+  every page load. `riskiestTool` makes a wrong class diagnosable without
+  re-deriving it. Its value is the FULL canonical tool name, while the card
+  renders only the segment after the plugin prefix. One load
+  showing `riskiest tool: memory-update` on the card and
+  `riskiestTool=mcp__plugin_memory_memory__memory-update` on this line is the
+  confirmation that both derive from one value.
+  **A non-empty `unresolved=` is the one to act on**: the agent
   is forced red, and the cause is that the built-in table or the registry
   mapping has drifted, not that the agent changed.
 - `op=risk-surface status=load-failed reason=<msg>` — the plugin registry did

package/payload/platform/plugins/admin/skills/whats-new/SKILL.md

@@ -9,6 +9,12 @@ Invoked by the admin agent directly.
 
 This is the platform's release timeline, newest first. Each entry shows the date it shipped and the version it shipped in, so you can tell the operator how current their install is. To compare, read the installed version from `capabilities-here` and match it against the versions below. Keep answers high level and in plain English; this is a summary, not a full commit log.
 
+## 2026-07-26 (0.1.509)
+
+- The Agents page now names the tool that set an agent's risk level, and shows it by its short name instead of the full internal path.
+- Publishing a site twice at the same time can no longer delete the live copy; the later publish now stands down instead of clearing the tree out from under the first.
+- A slow dashboard now records where the time actually went, so a sluggish load can be traced rather than guessed at.
+
 ## 2026-07-25 (0.1.508)
 
 - SiteDesk's screens now use a single, cleaner typeface throughout, and the old olive-green colour is gone.

package/payload/platform/plugins/cloudflare/bin/portal-index-push.mjs

@@ -122,7 +122,8 @@ export async function walkExposed(accountDir, exposed) {
  *           client: { query: (sql: string, params?: unknown[]) => Promise<Record<string, unknown>[]> },
  *           log: (line: string) => void, nowIso: string,
  *           exposeFolders?: string[], previousExposed?: string[] }} opts
- * @returns {Promise<{ exposed: string[], rows: number, schemaPresent: boolean }>}
+ * @returns {Promise<{ exposed: string[], rows: number, schemaPresent: boolean,
+ *                     generation?: number, superseded?: boolean }>}
  */
 export async function pushAccount(opts) {
   const { accountDir, accountId, client, log, nowIso } = opts
@@ -196,29 +197,26 @@ export async function pushAccount(opts) {
     [accountId],
   )
   const current = Number(state[0]?.currentGeneration ?? 0)
-  const next = current + 1
-
-  // Crash safety (Task 1923). A stage that died mid-batch — a D1 500 on one
-  // INSERT — leaves rows at `next` while the pointer stays at `current`. Every
-  // later run recomputes the same `next`, and its first colliding INSERT then
-  // hits UNIQUE(accountId, relPath, generation), so the index freezes at the
-  // stale generation until an operator hand-clears the orphans. Clearing `next`
-  // before staging lets a partial stage self-heal on the next cycle. Guarded on
-  // a non-zero count so the DELETE never precedes the flip on a clean cycle and
-  // the log line always means a recovered wedge. The count comes from a SELECT
-  // because the D1 client returns rows, not the affected-row meta.
-  const orphan = await client.query(
-    'SELECT COUNT(*) AS n FROM directory WHERE accountId = ? AND generation = ?',
-    [accountId, next],
+
+  // Claim a generation above EVERY generation this account already holds, not
+  // merely above the pointer (Task 1926). Two pushes of one account are
+  // reachable: the admin server's 60s loop and a hand-run `--account` are
+  // separate processes, so no in-process lock can see both. `current + 1` let
+  // the second writer claim the generation the first had just flipped onto and
+  // then delete it. A peer that has inserted even one row is visible in this
+  // MAX, so the two runs claim different generations.
+  //
+  // This also retires the Task 1923 pre-stage clear. That clear existed because
+  // a later run recomputed the SAME generation as a dead partial stage and
+  // collided on UNIQUE(accountId, relPath, generation); claiming above the
+  // orphans leaves nothing to collide with, and the post-flip sweep collects
+  // them. The clear was itself the DELETE a concurrent peer could aim at the
+  // live tree, so removing it removes the hazard rather than narrowing it.
+  const staged = await client.query(
+    'SELECT MAX(generation) AS g FROM directory WHERE accountId = ?',
+    [accountId],
   )
-  const orphanCount = Number(orphan[0]?.n ?? 0)
-  if (orphanCount > 0) {
-    await client.query('DELETE FROM directory WHERE accountId = ? AND generation = ?', [
-      accountId,
-      next,
-    ])
-    log(`${TAG} op=stage-clear account=${accountId} generation=${next} cleared=${orphanCount}`)
-  }
+  const next = Math.max(current, Number(staged[0]?.g ?? 0)) + 1
 
   for (const row of rows) {
     await client.query(
@@ -231,18 +229,56 @@ export async function pushAccount(opts) {
 
   // THE atomic point. Upsert rather than insert-or-update, so an account
   // publishing for the first time and one republishing take the same path.
+  //
+  // The WHERE moves the pointer FORWARD ONLY (Task 1926). Without it a writer
+  // that was overtaken while staging drags every reader back onto a generation
+  // the winner has already swept, which is an empty tree. SQLite has carried a
+  // WHERE on upsert's DO UPDATE since 3.24, and D1 is SQLite.
   await client.query(
     'INSERT INTO directory_state (accountId, currentGeneration) VALUES (?, ?) ' +
-      'ON CONFLICT (accountId) DO UPDATE SET currentGeneration = excluded.currentGeneration',
+      'ON CONFLICT (accountId) DO UPDATE SET currentGeneration = excluded.currentGeneration ' +
+      'WHERE excluded.currentGeneration > directory_state.currentGeneration',
     [accountId, next],
   )
+
+  // Read back rather than assume: the D1 client returns rows, not affected-row
+  // meta, so a suppressed flip is otherwise indistinguishable from an applied
+  // one.
+  const flipped = await client.query(
+    'SELECT currentGeneration FROM directory_state WHERE accountId = ?',
+    [accountId],
+  )
+  const pointer = Number(flipped[0]?.currentGeneration ?? 0)
+  if (pointer !== next) {
+    // A peer published while this run was staging. Its tree is live and
+    // complete, ours is invisible, and its rows are collected by the peer's
+    // next sweep. Sweeping or verifying from here would act on, and measure,
+    // the PEER's tree — reporting a failure where nothing failed. Logged rather
+    // than silent: a suppressed run is otherwise indistinguishable from one
+    // that never started.
+    log(
+      `${TAG} op=stage-contended account=${accountId} generation=${next} ` +
+        `pointer=${pointer} action=superseded`,
+    )
+    return {
+      exposed: r.exposed,
+      rows: rows.length,
+      schemaPresent: true,
+      generation: next,
+      superseded: true,
+    }
+  }
   log(`${TAG} op=flip account=${accountId} generation=${next}`)
 
   // Housekeeping only. A failure here leaves stale rows nothing reads, which is
   // waste rather than incorrectness — so it must not fail the publish that has
   // already succeeded.
   try {
-    await client.query('DELETE FROM directory WHERE accountId = ? AND generation != ?', [
+    // Strictly older only (Task 1926): `!= next` would delete a peer's HIGHER
+    // in-flight stage. Rows above `next` left by an abandoned stage still
+    // converge, because the next cycle claims a generation above them and
+    // sweeps them once it flips.
+    await client.query('DELETE FROM directory WHERE accountId = ? AND generation < ?', [
       accountId,
       next,
     ])
@@ -291,9 +327,13 @@ export function readState(platformRoot) {
 }
 
 /**
- * Written ONLY after op=verify, so a stale entry means the push stopped
- * succeeding rather than merely stopped logging. Read-modify-write per account
- * so one account's failure does not erase another's record.
+ * Written after a publish that reached the flip, so a stale entry means the
+ * push stopped succeeding rather than merely stopped logging. That includes a
+ * run superseded by a concurrent peer (Task 1926), which emits
+ * `op=stage-contended` instead of `op=verify`: the index WAS refreshed inside
+ * this cycle, by the peer, and recording otherwise would report a healthy
+ * account as never-ran. Read-modify-write per account so one account's failure
+ * does not erase another's record.
  * @param {string} platformRoot @param {string} accountId
  * @param {{lastPushMs: number, rows: number, exposed: string[]}} entry
  */