instar 1.3.229 → 1.3.230

package/upgrades/1.3.230.md

@@ -0,0 +1,66 @@
+# Upgrade Guide — vNEXT
+
+<!-- assembled-by: assemble-next-md -->
+<!-- bump: minor -->
+
+## What Changed
+
+Adds `instar dev:profile-node [pid]` — a contributor/agent dev command that
+CPU-profiles a **running** node process and prints its hottest JS functions
+(function + `file:line` + self-time %).
+
+**Why this exists (the technique that cracked a real bug).** macOS `sample` — and
+lsof snapshots, empty `gh run --log`, port-grep — cannot symbolicate a node
+process's **JS** frames; they show only native/V8 frames, so a busy agent server's
+actual hot *function* stays invisible. The way to see it is the node process's own
+introspection: `SIGUSR1` opens node's inspector on the running process, then a CDP
+CPU profile over the inspector websocket reports the exact JS call frames. This is
+literally how the systemic agent-server hot-loop was pinned — `StateManager.listSessions`
+burning ~30% of CPU in `readFileUtf8`, invisible to every other tool. This command
+turns that hard-won procedure into one reusable command.
+
+**What it does:** resolves the pid (or auto-finds the hottest `node` process) →
+`SIGUSR1` → probes `127.0.0.1:9229-9235` for the inspector target → captures a CPU
+profile (default 5s, `--duration`) → prints the top frames by self-time (`--top`).
+The biggest non-idle frame is the hot path. Read-only aside from the SIGUSR1, which
+opens the inspector on localhost for the process's lifetime (it closes on the next
+restart) — the command says so in its output.
+
+Pairs with `dev:ci-failures` and `dev:preflight` as the contributor dev-loop
+toolkit (friction → tooling).
+
+## What to Tell Your User
+
+Nothing required — it is a developer/agent tool for working on Instar. When an
+agent server is mysteriously burning CPU, the instar dev:profile-node command
+prints the exact JS function responsible, which native profilers cannot show.
+
+## Summary of New Capabilities
+
+- `instar dev:profile-node [pid] [--duration <sec>] [--top <n>]` — CPU-profile a
+  running node process (SIGUSR1 + inspector + CDP) and print its hottest JS frames.
+  No pid → profiles the hottest node process.
+
+## Evidence
+
+- Built directly from the technique that pinned the real hot-loop: after `sample`,
+  lsof, grep, and log-diffing all failed, SIGUSR1 + a CDP CPU profile showed 30.8%
+  `readFileUtf8` + 7% `listSessions` (StateManager) — the JS frame nothing else
+  could see — leading to the cache fix.
+- Live end-to-end smoke: profiling a spawned CPU-busy node correctly reported the
+  hot frame (`listOnTimeout`, where the busy loop ran).
+- Tests: `tests/unit/devProfileNode.test.ts` — pure `aggregateHotFrames` (ranking,
+  file:line normalization, anonymous-frame fallback, empty profile, topN) +
+  `findInspectorTarget` (first-hit / none) + `runDevProfileNode` orchestration with
+  injected deps (happy path, no-pid→hottest, no-node→exit1, signal-fail→exit1,
+  no-inspector→exit1, capture-throw→exit1).
+
+## Also in this change
+
+Fixes a real false-negative in `scripts/docs-coverage.mjs`: a command's coverage
+was keyed on its camelCase **file** name (`devProfileNode`), so the needle
+`instar devProfileNode` never matched the shipped colon-command
+`instar dev:profile-node` — every `dev:*` command (ci-failures, preflight,
+profile-node) read as undocumented. The matcher now also tries a kebab form and
+treats `:`/`-` as interchangeable, so colon-commands count. (The new command
+surfaced this — friction → structural fix.)

package/upgrades/dev-profile-node-tool.eli16.md

@@ -0,0 +1,42 @@
+# A stethoscope for a node process that's running hot
+
+## The one-sentence version
+When one of our programs is burning the CPU for no obvious reason, the usual tools can tell you *that* it's busy but not *what line of code* is doing it. This new command — `instar dev:profile-node` — presses a stethoscope to the running program and prints the exact function that's hogging the CPU.
+
+## The problem it solves
+Picture a car idling in the driveway but somehow guzzling gas. You pop the hood with your usual gauge — it says "engine's working hard" but won't tell you *which part*. That's macOS's built-in profiler on a Node.js program: it sees the engine block (the low-level machinery) but can't read the actual code, so the real culprit stays hidden.
+
+This bit us for real: three of our agent servers were each burning ~50–60% CPU, bouncing them never helped, and four different tools all came back "busy, but I can't tell you why." 
+
+## The trick that worked (now a one-liner)
+Node programs have a hidden superpower: send them a specific nudge (the SIGUSR1 signal) and they open a little diagnostic port on themselves. Connect to that port, record a few seconds of "what function is running right now," and it tells you — in plain code terms — exactly where the time goes.
+
+That's how we finally caught it: **30% of the CPU was in `readFileUtf8`, inside `listSessions`** — a function re-reading every session file off the disk, over and over. Invisible to everything else; obvious once we looked through the right lens.
+
+`instar dev:profile-node` bakes that whole trick into one command:
+- Point it at a process (or let it auto-find the hottest one).
+- It nudges the process, records ~5 seconds, and prints the hottest functions with their percentages.
+- The biggest non-idle line is your culprit.
+
+## What it looks like
+```
+Hottest JS frames (self-time) for pid 12673:
+   87.6%  listOnTimeout  node:internal/timers:546
+    9.9%  (program)
+    2.5%  (garbage collector)
+```
+(That's from a test process running a busy loop inside a timer — and it correctly fingered the timer callback.)
+
+## The safety notes
+1. The only thing it changes is opening that diagnostic port — and only on **localhost** (your own machine), only until the process next restarts. The command tells you it did this.
+2. It **samples** the process — it never kills it, restarts it, or edits anything.
+3. It's a developer tool, run on demand — nothing in the running system uses it.
+
+## Why it matters
+The hard part of fixing a mystery slowdown isn't the fix — it's *finding* the cause. This turns a clever, hard-won debugging trick into a reusable one-liner, so the next time any node process runs hot, the answer is one command away instead of an afternoon of dead ends. (Same spirit as our `dev:ci-failures` tool: when a workaround saves the day, make it permanent.)
+
+---
+
+**Rendered (verified HTTP 200):** https://echo.dawn-tunnel.dev/view/e5433071-bf6f-4cf0-b97a-5e16d9f7b160?sig=2c015aaeedc96ebb20d57b326f44d88eaa93dd2bcba0b4e8347a33d23f4d2b4b
+
+*See also: the full success-story + meta-insights writeup for this run.*

package/upgrades/side-effects/dev-profile-node-tool.md

@@ -0,0 +1,85 @@
+# Side-Effects Review — `instar dev:profile-node`
+
+**Version / slug:** `dev-profile-node-tool`
+**Date:** `2026-06-03`
+**Author:** `echo`
+**Second-pass reviewer:** `not required (dev-only CLI; read-only sampling; no server/runtime surface)`
+
+## Summary of the change
+
+A new dev CLI command that CPU-profiles a running node process via SIGUSR1 + the
+node inspector + a CDP CPU profile, and prints the hottest JS frames. New file
+`src/commands/devProfileNode.ts` + a cli.ts registration + a manifest entry +
+awareness lines. No server, route, config, or runtime-behavior change.
+
+## Decision-point inventory
+
+1. **pid resolution** — explicit `[pid]` digits win; otherwise auto-pick the
+   hottest `node` process (`ps -Aceo pcpu,pid,comm`). No node found → exit 1.
+2. **Inspector discovery** — probe ports 9229–9235 (node's default + increments
+   when the default is taken); first target with a `webSocketDebuggerUrl` wins;
+   none → exit 1 with a clear message.
+3. **Profile aggregation** — `aggregateHotFrames` sums per-node `hitCount` by call
+   frame, normalizes the path to `dist/…`/`src/…`, +1 to the 0-based line, and
+   returns top-N by self-time. Idle/native frames are kept on purpose (seeing
+   "48% idle, 30% readFileUtf8" is the signal).
+4. **Exit codes** — 0 on a successful print (even "no samples"); 1 only on an
+   operational failure (no node, signal failed, no inspector, capture threw).
+
+## 1. The one real side effect — SIGUSR1 opens the inspector
+
+SIGUSR1 tells node to start its inspector on `127.0.0.1` (localhost-only) for the
+life of the process. Implications, all bounded:
+- **Localhost-only** — the inspector binds 127.0.0.1, not a public interface; it is
+  not remotely reachable.
+- **Transient** — it closes when the process restarts; it does not persist.
+- **Disclosed** — the command prints that it left the inspector open on the pid.
+- **No state mutation** — profiling samples the process; it does not change its
+  behavior, memory, or files.
+
+This is the same standard procedure node documents for live debugging; it is the
+*only* way to symbolicate JS frames on a running process. The benefit (pinning a
+hot loop that no other tool can see) outweighs a transient localhost debug port on
+a dev box. Operators who object can simply not run the command.
+
+## 2. Over/under-block
+
+Not applicable — it gates nothing and changes no behavior. A wrong pid or a
+non-node target fails cleanly with exit 1; it never acts on the target beyond
+SIGUSR1 + reading the profile.
+
+## 3. Blast radius
+
+Zero runtime surface. The command runs only when a contributor/agent invokes it.
+No server code path imports it; it is a leaf CLI command behind a dynamic import
+(like `dev:ci-failures`). The `ws` dependency is already present (`^8.19.0`).
+
+## 4. Migration parity
+
+CLAUDE.md template (`generateClaudeMd`) + `site/.../cli.md` + the builtin-manifest
+(`cli:dev-profile-node`) are updated in lockstep so the awareness + discoverability
+checks stay green. No config/hook/skill migration — it is a pure CLI addition
+(same shape as `dev:ci-failures`, which needed none).
+
+## 5. Testability
+
+`aggregateHotFrames` and `findInspectorTarget` are pure/dep-injected; the
+SIGUSR1 / inspector-http / ws-CDP boundary is behind `ProfileNodeDeps` so the
+orchestration is unit-tested without a real process. A live end-to-end smoke
+(profiling a spawned busy node) confirmed the real path.
+
+## 6. What it does NOT do
+
+- Does not kill, restart, or modify the target (only SIGUSR1 + sample).
+- Does not run as part of the server or any job — invoked on demand only.
+- Does not expose a remote/public debug port (inspector is localhost).
+
+## 7. Collateral fix — docs-coverage matcher
+
+Adding the command surfaced a false-negative in `scripts/docs-coverage.mjs`: it
+matched a command by its camelCase file name, so colon-commands (`dev:*`) never
+matched their documented form and read as undocumented (the floor was sitting
+right at the limit, so the new command tipped it). The matcher now also tries a
+kebab form and normalizes `:`↔`-`. Audit-only script (the weekly docs-coverage
+report); no runtime/behavior effect. It re-classifies the three `dev:*` commands
+as documented, which is correct (they ARE in cli.md + the CLAUDE.md template).