@bompus/house-rules 0.0.0-stage → 0.13.0

package/skills/hr-benchmarking/NOTICE.md

@@ -0,0 +1,36 @@
+# Source notices
+
+This notice accompanies the hr-benchmarking bundle when copied on its own.
+
+## cursor/plugins (pstack)
+
+Source: https://github.com/cursor/plugins (compared at revision `e43c7ee`)
+
+- `skills/hr-benchmarking/SKILL.md`: the validity checks are adapted from
+  `pstack/skills/benchmark-checklist/`.
+
+Copyright (c) 2026 Lauren Tan
+
+## house-rules and MIT permission notice
+
+MIT License
+
+Copyright (c) 2026 Aaron Queen
+
+Permission is hereby granted, free of charge, to any person obtaining a copy
+of this software and associated documentation files (the "Software"), to deal
+in the Software without restriction, including without limitation the rights
+to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
+copies of the Software, and to permit persons to whom the Software is
+furnished to do so, subject to the following conditions:
+
+The above copyright notice and this permission notice shall be included in all
+copies or substantial portions of the Software.
+
+THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
+IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
+FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
+AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
+LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
+OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
+SOFTWARE.

package/skills/hr-benchmarking/SKILL.md

@@ -0,0 +1,279 @@
+---
+name: hr-benchmarking
+description: Benchmark equivalent work with repeated timing, CPU and memory measurements under stated inputs and controls, and check whether saved results support a performance claim. Use when asked to benchmark, to measure which option is faster or uses less memory, or to time agent or model runs. Not for public model leaderboard research.
+---
+
+# Benchmarking
+
+Follow the repository and host's commands, resource limits, authorization and
+landing rules. Reuse an existing workload and runner before building a harness.
+Read [the tool reference](references/tools.md) when selecting a benchmark,
+profiler or correctness check, or interpreting its measurements.
+
+## Define the comparison
+
+State the question, baseline, candidate, representative input and timed region.
+Exercise ordinary supported inputs and cases that disable an optimization.
+State any restrictions behind a performance claim.
+
+For a requested candidate set, record its inventory and distinguish
+configurations or capacity modes. Keep untested, invalid and unsupported
+candidates visible with reasons. A subset comparison establishes results only
+for its tested scope, not a winner across the full inventory.
+
+Test the newest stable release of each candidate. When the version installed
+or deployed differs, say so in the results, and ask which to test if the
+request does not settle it. When told to use an older version, use it and
+offer the newest as an alternative.
+
+Before running, list the selected engines, workloads, metrics, controls and
+repetitions. Count the planned runs explicitly, including controls and stages;
+state which decision each retained dimension helps make. Choose combinations
+for that decision rather than automatically running every combination. Use a
+broader matrix when its required coverage is stated. State the benchmark's
+expected wall-clock time, waits included, before the first run.
+Name the measurement unit: request, operation, batch or whole command. State
+whether setup, startup, compilation, input loading, retries and cleanup count.
+
+Record exact runtime, compiler, dependency and runner versions, build flags,
+input identity and source revisions. Freeze copied quality and control artifacts;
+record their hashes and the executed artifact's identity. Agreement across
+runtimes alone does not verify the intended artifact. Keep these fixed across
+repeated arms; reject mixed versions when resuming a checkpoint.
+
+Compare runtime knobs within the same build. Use equivalent outputs, release
+settings and comparable tuning. An untuned arm compares configurations rather
+than implementations.
+
+Keep warm and cold conditions explicit. Reset state or use fresh processes
+where the question requires independence. Record warmup and cache conditions.
+A reused service needs equivalent state across arms, not an arbitrary restart.
+
+## Confirm the work
+
+Validate outputs, consumed results and error counts outside the timed region
+where practical. Confirm requests arrived and asynchronous work was awaited.
+Include failures, timeouts and unresolved attempts in both arms' denominators.
+A faster wrong result or skipped operation is not a performance improvement.
+
+Identify the limiting resource from CPU versus wall time, resource counters or
+a separate profile. Check that throughput is possible given the machine and
+input. When the generator limits the result, report that limit.
+
+Before changing a component to remove a suspected cost, attribute time to that
+component in the real workload with a profile or a counter. For fast feedback,
+benchmark each changed piece through its actual production entry point with
+representative fixed inputs and equivalent consumed outputs. Do not benchmark a
+copied implementation or stub out work material to the comparison.
+
+Compare each piece before and after against its immediate predecessor. Before
+running, declare integration milestones when changed pieces enter the consuming
+workload. Keep that workload before any piece changes as the original baseline.
+At those milestones, compare the integrated workload with both its immediate
+predecessor and the original baseline to track cumulative effects. Plan applicable
+caller and integration checks for copying, serialization, startup and orchestration
+costs that isolation excludes. Run those checks at the milestones and report
+results or specific coverage limits. Re-time the change in the workload that uses
+it at those milestones; a full-application run is not required after every piece.
+An isolated gain does not establish an end-to-end gain. A setting that ran 7.6
+times faster in isolation changed a 33 s indexing run by 0.1 s.
+
+Select metrics for the question and limiting resource: wall time or latency,
+memory, disk I/O, CPU, throughput, errors and output size as relevant. State each
+metric's unit, process scope and interval; distinguish peak process memory from
+retained memory, logical bytes from physical storage I/O, and warm from cold cache
+state. Cache hits and delayed writeback can hide storage costs; record unavailable
+counters and attribution limits rather than inferring physical I/O from logical
+bytes. Do not require irrelevant additional counters; retain the CPU and memory
+observations required below.
+
+## Measure without changing the experiment
+
+Keep measured files, toolchains and services unchanged, and keep other work on
+the machine from competing with the run.
+
+Prefer removing competing work to raising priority. Record effective CPU and
+I/O scheduling policies and memory protections; keep them equal across arms.
+Use normal scheduling and all available CPUs for ordinary measurements. Omit
+automatic `nice` and `taskset` wrappers; verify inherited priority and affinity
+rather than assuming their omission resets either. Use restricted affinity or
+changed priority only for representative deployment settings, an explicit resource
+agreement or a stated sensitivity question. Record the reason and effective values.
+Preserve frozen protocols and report constrained results within their tested scope.
+Read [scheduling controls](references/tools.md#scheduling-controls) before
+choosing them.
+
+Apply worker limits to requested processing concurrency. Report sampled OS-thread
+counts separately; runtime background threads are not processing workers. A
+thread count alone does not establish CPU contention; keep it diagnostic unless
+the experiment separately justifies using it to reject a run.
+
+Before collecting results, set the criteria for rejecting a run: competing
+load, swapping and memory-limit or OOM events, each with its metric and
+threshold. Check the machine against them before each arm and watch them
+throughout the measured interval. A run that fails them, or whose inputs change, is rejected even when the benchmark caused the
+failure. A rejection invalidates that measurement; it does not by itself show
+an implementation defect. Keep rejected runs outside the accepted sample and
+report them; a later passing run neither erases a rejection nor loosens the
+criteria.
+
+Attribute CPU, load and resident memory to the recorded benchmark process tree
+or resource scope before calling them competing work. Expected benchmark
+utilization is not contention, and total host CPU and load average include it.
+Keep an activity counter diagnostic until separate evidence shows its cutoff
+matters. Collect boundary counters adjacent to the measured work, with
+unrelated probes outside that interval, and record sample times. A counter
+interval extending past completion cannot by itself establish overlap; keep
+post-run observations diagnostic and let the machine settle before the next arm.
+
+Before claiming negligible observer impact or a small decisive gain, test
+sensitivity to observation separately with balanced controls and a preset
+precision criterion. Retain inconclusive controls; do not subtract observer CPU
+from workload wall time. For remote agent or model evaluations, record the
+timing regime and concurrent local work.
+
+Screen correctness and quality first, then use one run per selected arm to
+explore timing and resource behavior. Preserve screened-out results and their
+reasons. Label screening exploratory; one run does not support a performance
+claim. A correctness-only check can use one candidate run; report its coverage.
+
+Before exploratory runs, define the decision-relevant signal for selecting
+comparisons for confirmation. Choose comparisons that meet that signal, plus
+decision-critical comparisons, for repeated confirmation; keep necessary controls.
+Set the confirmation plan before its runs and update the explicit run count
+when the scope changes.
+
+The default stopping rule for a performance claim is 3 runs per arm. Add 2 more
+per arm only when the arms' ranges overlap or the effect is near the noise,
+meaning the gap is within the variation criterion set below. 5 per arm is the
+cap. If 5 runs per arm leave the comparison unresolved, report it as
+inconclusive rather than adding runs. Use another count only when the user asks
+for it or a run is too long to repeat 3 times, and declare it before the first
+run, with the reason. Alternate the arms in every run so drift reaches each arm.
+
+For a performance claim, assess run-to-run variation from those alternating runs.
+Treat variation as already known only when earlier repeated runs of the
+same variant, build and input used the same workload, host, run conditions and
+variation statistic, and met the repetition count or stopping rule declared for
+this claim. Otherwise include an A/A arm,
+which runs the same variant twice in the same alternation. Its spread is that
+comparison's own noise floor, and it can supply the preset variation statistic.
+Before the first run used in that claim, set a reproducible variation criterion
+(statistic and threshold), and the stopping rule if it differs from the default.
+
+For short or tail metrics, such as the longest stall in a run, give the
+threshold an absolute floor or use a percentile or count metric; a percentage
+of a single small maximum can fail on the workload's own variation.
+Retain every result. Keep pilots that informed the plan separate from the
+claim's sample. Capture raw results, exit status, elapsed time, CPU and memory
+observations. Report monitoring overhead separately from the
+measured workload; batch compatible probes when they dominate elapsed time.
+Report the selected repetition count or stopping rule, variation statistic and
+threshold, sample count, median and range. Always give the run count and the
+range with any central value, never a bare average. Apply the preset
+criterion and report a gap within run-to-run variation as no clear change. Profile
+separately from the runs used to claim a speed difference.
+
+When using saved evidence for a performance claim, bind archived inputs, source,
+cohort and results to their recorded identities and artifact hashes. Validate
+historical reports against that frozen archive; later changes in the live
+checkout alone do not invalidate historical evidence. Verify run identities and
+stages against the declared plan.
+
+Recompute qualification from recorded raw counters and preset limits; an accepted
+summary or changed label is insufficient. Reject missing, duplicate or
+misclassified required observations. When an evidence reader exists, put these
+checks and focused corruption regressions there. After a rebase or merge
+touches the reader or a check it imports, run the integrated reader over the
+saved archive before landing, and compare its accepted and excluded records
+with the frozen reader's. A shared check changed on the base branch can reject
+records that the frozen reader accepted.
+
+Record memory by what it measures. Whole-process peak RSS includes startup,
+inputs and dependencies. Heap snapshots describe retained objects; allocation
+profiles describe allocation activity. Forced-GC probes are separate from
+ordinary latency measurements. Match retention semantics and payloads when
+comparing caches; equal requested capacities need not mean equal memory use.
+For cache-retention probes, keep the cache owner reachable through the final
+observation; otherwise collection of the owner can be mistaken for release by
+the cache operation.
+
+For WeakRef-based retention probes, use bounded allocation batches and calibrated
+collection boundaries after creating or dereferencing targets. Avoid observer
+arrays or locals that keep measured values strongly reachable; preserve the
+intended owner lifetime. Report live payload, cache occupancy and process RSS
+separately. Keep this collection protocol outside ordinary latency runs; yielding
+or forced collection alone does not guarantee that a target is reclaimed.
+
+Batch latency percentiles describe batches, not individual requests.
+
+## Report and decide
+
+Keep the full workload, versions, conditions, repetitions, output identity,
+errors and measurement units in the measurement record. Include excluded costs,
+coverage limits, collector overhead, scheduling, resource caps, pre-run
+machine checks and rejected attempts there.
+
+Pull requests and other change summaries show the workload, baseline and
+candidate, measured results, relevant checks and limits that affect the claim.
+Omit operational measurement details unless the reader needs them to interpret
+the result or asks for them. A claimed difference with unknown limits, unequal
+work or missing repetitions is unverified or inconclusive.
+
+Show numeric before/after results in a table with a benefit column containing
+percent change or a times improvement. Use the metric's direction and label
+worsening, unchanged or inconclusive results. With a zero or missing baseline,
+show an absolute change or state that the benefit cannot be calculated.
+
+There is no minimum gain size. A favorable difference counts as a gain when it
+lies outside the variation criterion declared before the first run (the comparison's
+own run-to-run spread, from its A/A arm or already-known variation, as a
+percentage of the baseline) and the outputs are identical. Report a gap inside
+that criterion as no clear change, and name the statistic and threshold beside
+it. An unfavorable difference outside it is a regression; label it worsening.
+Report memory and CPU costs beside the gain. If the variation criterion cannot support a percentage, keep its result
+separate and label unsupported differences inconclusive. Changing presentation
+does not change frozen acceptance or rejection gates.
+
+Show a comparison of three or more variants or conditions as a chart beside the
+before/after table, unless every row is no clear change. Choose the chart for the
+question:
+
+| Question                         | Chart                                      |
+| -------------------------------- | ------------------------------------------ |
+| Repeated runs of each variant    | Dot plot of every run, mean marked         |
+| Conditions crossed with variants | Grouped bars, baseline as a reference line |
+| Ranking of options               | Sorted horizontal bars                     |
+| Resource cost (CPU, memory)      | Bars per variant, one panel per resource   |
+| Change with load or time         | Line chart                                 |
+
+Put units and run counts on the axes, start bars at zero, mark a gap inside the
+variation criterion as no clear change, and give the chart an explicit background colour so
+it reads the same on a light or a dark page.
+In a pull request, the headline chart and when it may be skipped come from
+`hr-writing-pr`; this table applies to any other chart.
+
+For agent, model, rule or skill comparisons, distinguish development cases
+from held-out cases and report runs per case. Check each case for a ceiling: a
+case every arm already passes cannot show a pass-rate improvement. Report it
+apart from the cases that could, and report any metric the case does not
+saturate (time, cost, graded quality) on its own. Include a one-line instruction
+control when claiming an improvement from a rule or skill. Keep helper,
+setup and retry costs visible. Before collecting results, define a per-case
+contribution threshold. If a case exceeds it, report results both with and
+without that case.
+
+For bug-review evaluations, freeze the supported input contract and label
+procedure before calls. A passing reference suite does not certify a clean
+control. Adjudicate additional findings against that contract. A confirmed extra
+defect makes the affected specificity estimate unavailable; retain earlier
+receipts and introduce a new control version. Leave disputed domains unresolved.
+
+Report provider completion, output-format compliance and independently checked
+artifact acceptance separately. Distinguish observed call time from verification
+and rework time. Keep returned usage units separate from attributable account
+charges; unknown attribution remains unknown.
+
+Record every experiment, rejected ones included, with the observed result and
+what was decided. End with adopt, reject or inconclusive, and the evidence for that
+decision. Any implementation or publication still follows the user's scope.

package/skills/hr-benchmarking/agents/openai.yaml

@@ -0,0 +1,2 @@
+interface:
+  display_name: "HR: Benchmarking"

package/skills/hr-benchmarking/references/javascript.md

@@ -0,0 +1,130 @@
+# JavaScript and TypeScript measurement
+
+Use built-in runtime profilers and oha from [the tool reference](tools.md).
+TypeScript applications execute through a runtime; compiler timing and
+application timing answer different questions.
+
+## TypeScript measurement
+
+Use [compiler diagnostics](https://www.typescriptlang.org/tsconfig/extendedDiagnostics.html)
+for compiler phases and repeated whole-command timing for builds. For application
+performance, record the executed artifact, build target, module format and source
+maps. Use the same compiled JS artifact for an engine-only comparison; report
+startup or transformation separately when it is part of the question.
+
+[Node type stripping](https://nodejs.org/api/typescript.html) performs no type
+checking and ignores `tsconfig.json`. A successful direct `.ts` execution does
+not establish TypeScript correctness.
+
+## Running under Bun versus Node
+
+Confirm the runtime before comparing: print `process.execPath` or
+`process.versions.bun` from inside the code under test, in worker threads too.
+Verified on Bun 1.4.x:
+
+- `bun x vitest` is not Bun. `bun x` honors the tool's `#!/usr/bin/env node`
+  shebang and runs the whole suite under Node. Use `bun --bun x vitest run`.
+- `node` can resolve to Bun. On Bun 1.4.3, a package script that calls `node` printed Bun's
+  version under `bun --bun run` and Node's under plain `bun run`; a `Bun.spawnSync(['node', …])`
+  reached real Node in both modes. The shim depends on Bun's `run.bun` setting, which defaults
+  on only when `node` is absent from `PATH`. Check `process.versions.bun` in the spawned process,
+  and filter `PATH` entries matching `/bun/i` when you need real Node.
+- With no `node_modules` up the tree, `require.resolve` falls back to Bun's global
+  install cache ([oven-sh/bun#42893](https://github.com/oven-sh/bun/issues/42893)).
+  Pass `--no-install` for a hermetic "not installed" result.
+- File-path and `{ eval: true }` workers both run under Bun, and `node:sqlite` and
+  NAPI `.node` addons load.
+- Many busy, allocating workers wait on each other's parallel GC marking
+  ([oven-sh/bun#44186](https://github.com/oven-sh/bun/issues/44186)); with 8
+  workers `BUN_JSC_numberOfGCMarkers=1` roughly halved wall time. Attribute with a
+  profiler before concluding anything about worker scaling.
+- Node 26.11 renamed its synchronous SQLite classes to `Database` and `Statement`
+  (`DatabaseSync` and `StatementSync` stay as aliases) and caches common property
+  keys in its SQLite bindings. Record the exact Node version beside SQLite
+  results; an older baseline does not establish the gap on the new runtime.
+
+## Choosing what to hunt
+
+Start with something a user waits for. Use a representative workload and data size, then count the work before timing individual functions.
+
+- Count messages and bytes per action, SQL statements, subprocesses, React commits, queue depth and oldest-item age. Measure idle CPU, event-loop delay and utilization. Use a CPU profile to attribute expensive work; delay alone does not identify its cause.
+- Check repeated work first: whole-state rebuilds after small updates, serialization per subscriber, repeated parsing or validation, N+1 queries and per-row transactions. Check for backpressure when producers outrun consumers.
+- Vary one input size while holding the rest fixed. Look for quadratic growth, large linear costs, and fixed costs paid too often. A high count alone does not establish a bottleneck.
+- Check work that survives its owner: listeners, timers, caches, pending requests and reconnect loops. Measure repeated open/close cycles and hidden, idle and disconnected states separately.
+- Profile startup imports, rendering and layout, compression, logging and tracing when they appear in the user-visible path. Investigate engine-specific tuning only when a profile identifies the relevant hot loop.
+
+Record null results and their tested conditions. Stop a candidate when its measured benefit is within the declared variation criterion, or when it cannot materially improve the chosen user-visible outcome. Do not use a universal percentage cutoff.
+
+### Deciding whether to move synchronous work
+
+Prioritize hot paths, then inspect less frequent paths for long individual stalls. A rare operation can still delay unrelated requests.
+
+Measure the longest blocked interval and the average blocked milliseconds per call multiplied by calls per second. The product is milliseconds blocked per second; divide by 1,000 to estimate the duty-cycle fraction. This average does not describe burst latency. Compare against the application's actual response and frame budgets. A 60 Hz frame is about 16.7 ms; higher refresh rates leave less time.
+
+Reduce unnecessary work first. Consider reuse, bounded caching, batching and yielding between chunks when their correctness requirements fit. A promise or `async` wrapper does not make CPU work leave the event loop.
+
+Use the runtime's asynchronous APIs for supported I/O. They add scheduling and completion overhead and can contend for a shared thread pool. For CPU work, compare chunked yielding with a reused [worker or worker pool](https://nodejs.org/api/worker_threads.html). Include worker startup, message and clone or transfer costs, queueing, memory, cancellation and shutdown. Do not create a new worker for every small call by default.
+
+Compare equal completed work under representative concurrency. Report wall time, CPU, memory and unrelated-request latency beside the longest stall. Offloading may improve responsiveness while costing more total CPU; state that tradeoff. A saturated worker queue can erase the improvement. There is no universal payload size or duration at which async or a worker wins.
+
+### Runtime checks
+
+- **Cloudflare Workers:** [deployed timers](https://developers.cloudflare.com/workers/runtime-apis/performance/) such as `Date.now()` and `performance.now()` advance after I/O, so they cannot time an uninterrupted CPU section. Local Wrangler uses workerd and permits advancing timers; use it for attribution and confirm deployed behavior with platform CPU metrics. Check current platform limits. A Durable Object runs JavaScript on one thread, but requests can interleave across awaits; distinguish synchronous blocking from storage gates and explicit concurrency controls.
+- **Bun:** JavaScriptCore and V8 can have different hot paths. Verify the runtime and use its profiler; remeasure relevant changes on each supported runtime.
+- **Electron:** attribute renderer and main-process work separately. Synchronous IPC blocks the caller; expensive main-thread work can delay all windows.
+- **React Native and web:** use production builds. Count commits per actual streamed update or action, not per estimated token. Separate JavaScript work from layout, paint and native UI work. Record visibility and observer overhead.
+
+## Event-loop and server-code measurement
+
+Each event-loop thread of a Node or Bun server runs request handling, timers and
+synchronous database calls. A slow call matters when it holds that thread, so
+measure the hold, not only the duration. (M) marks a result measured on one
+workload and host, not a benchmark.
+
+- **Measure the longest stall, not wall time.** Run a 1 ms `setInterval` that
+  records the largest gap between ticks while the call runs, after letting the
+  loop settle first. Report it beside wall time. They differ: a snapshot read took
+  79 ms of wall time and held the loop for 12.6 ms because the runtime yielded
+  between steps (M). `perf_hooks.monitorEventLoopDelay` has a resolution of at
+  least 1 ms, so use it for coarse trends, not for claims of a few ms. State the
+  probe interval beside any figure near it.
+- **Time the real function, not a copy.** A copied string-appending coalescer
+  looked quadratic; the real one showed equal CPU and equal longest stall within
+  noise, because V8 joins repeated concatenation lazily (M). A replica that leaves
+  out framework overhead is only a lower bound. Run server code inside the
+  project's own test runner with its in-memory test layer, write results to a file
+  because runners often swallow `console.log`, and delete the temporary test
+  afterwards.
+- **Count volume when volume is the cost.** For code that emits events or
+  messages, feed a fixed synthetic input through the real component and count the
+  events and serialized bytes at several sizes. Counts are deterministic, so they
+  need no repetitions; stall and wall time still do. One adapter emitted three
+  events per streamed chunk, two of them carrying the whole reply so far: 15,003
+  events (three per chunk plus one closing update of each) and 263.6 MB for 5,000
+  chunks of 10 characters (about 125 MB for each of
+  the two full-text copies), growing about 3.3 to 3.6 times per doubling, with a
+  longest stall under 6 ms (M). That is a volume problem, not
+  a stall.
+- **Separate loop time from child-process time.** Work done by an asynchronously
+  spawned process uses CPU but does not hold the loop. Count such a spawn as loop
+  cost only for its spawning and parsing. `spawnSync`, `execSync` and `execFileSync` block the
+  loop for the child's whole lifetime, so count that time as a stall.
+- **Prove that work left the loop with a test that cannot pass otherwise.** Hold
+  the main connection or thread busy (blocked on a latch) and
+  assert the moved work still completes. A speed measurement alone cannot show
+  where the work ran. Moving one read off the loop took the longest stall from a
+  median 165 ms to 2.5 ms over 12 runs per arm, with no clear change in wall time, measured with a 2 ms probe (M).
+- **Measure retention with forced garbage collection.** Run N synthetic sessions
+  or turns, force a collection, and compare heap deltas with the suspect field
+  kept and dropped, keeping the owner reachable to the end. One researcher's run, not
+  repeated, kept only the two fields that were read and went from 61.45 MiB to
+  2.10 MiB over 30 sessions of 10 turns of 200 KiB.
+- **Time startup on the built artifact, interleaved.** Alternate arms on
+  one build, using the default run count in [the skill](../SKILL.md). An
+  environment variable can stand in for an API
+  call, but note what it changes (it also reaches child processes). A V8 compile
+  cache took a CLI's `--help` from a 554 ms median to 381 ms over 11 runs (M).
+- **Treat another researcher's numbers as leads.** Of three top candidates from a
+  model's blind read-only pass, re-measuring left one refuted, one mostly covered
+  by existing work and one real but small. Re-run a lead with the real function
+  and the stall probe before acting on it.

package/skills/hr-benchmarking/references/tools.md

@@ -0,0 +1,165 @@
+# Benchmarking and investigation tools
+
+These defaults come from tools used in our work. Prefer a suitable runner
+already used by the project. Check the official version's documentation before
+relying on flags or interpreting output.
+
+## Timing and resource measurement
+
+| Task                               | Default and reason                                                                                                                                     | Limits to record                                                                                                                                                                                                                                                                                                                                                                 |
+| ---------------------------------- | ------------------------------------------------------------------------------------------------------------------------------------------------------ | -------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- |
+| Whole CLI commands                 | [Hyperfine](https://github.com/sharkdp/hyperfine): repeated runs with wall time, CPU time and peak memory, warmup/preparation and JSON output.         | Works across Linux, macOS and Windows. Declare warm/cold conditions. Since 2.0 commands run without a shell unless `-S` or `--shell` is given; 1.x needs `-N` for that. Peak memory and `--metrics` hardware counters are measured on Linux and macOS only; on Windows use another tool for memory. Check `schema_version` before reading exported JSON; 2.0 changed its layout. |
+| JavaScript operations              | [Tinybench](https://github.com/tinylibs/tinybench), directly or through the project's benchmark runner: warmup and task latency/throughput statistics. | The unit is a task invocation. For 1,000-operation batches, batch p99 divided by 1,000 is the p99 of batch-average costs, not mean operation cost or individual-operation p99.                                                                                                                                                                                                   |
+| HTTP load, including JS/TS servers | [oha](https://github.com/hatoo/oha): a native load generator with concurrency/rate controls, latency distributions and JSON output.                    | Declare protocol, connections, rate, deadlines, keepalive and error handling. Check generator CPU/network limits. For fixed-rate load, `--latency-correction` corrects coordinated omission; it is ignored unless `-q` is set.                                                                                                                                                   |
+| Command CPU and memory             | [GNU time](https://www.gnu.org/software/time/manual/time.html), commonly `/usr/bin/time -v`: wall time, CPU and maximum RSS.                           | GNU options differ from shell `time` and BSD/macOS tools. RSS is a lifetime high-water mark for the observed process; it does not isolate retained cache memory.                                                                                                                                                                                                                 |
+| Host or process contention         | Linux [mpstat/pidstat](https://github.com/sysstat/sysstat): processor utilization and process CPU, memory and I/O observations.                        | Sampling can miss short bursts; some counters need kernel support. These explain interference, not application latency.                                                                                                                                                                                                                                                          |
+
+On Linux with compressed swap enabled, observe compressed-swap activity as
+well as disk swap I/O. Unchanged `pswpin`/`pswpout` counters do not establish
+absence of swapping. For [zswap](https://www.kernel.org/doc/html/latest/admin-guide/mm/zswap.html),
+record available `zswpin`/`zswpout`/`zswpwb` deltas from the benchmark cgroup's
+`memory.stat` over the measured interval. Label host-wide deltas as system-wide;
+they cannot attribute activity to the workload.
+
+Use [cgroup swap and zswap accounting](https://www.kernel.org/doc/html/latest/admin-guide/cgroup-v2.html)
+(`memory.swap.current`, `memory.zswap.current`) to distinguish the workload
+from other jobs. These occupancy values do not measure activity by themselves.
+Record missing counters and preserve the run plan's declared pressure criteria.
+
+## Process attribution
+
+Prefer `pidstat -u -r -d -p ALL 1 20` for the standard Linux process view.
+For receipts that retain PID birth identity, use the bundled collector:
+
+```sh
+python3 scripts/process-attribution.py --seconds 20 --output /disk/path/receipt.json
+```
+
+Run from the skill directory. Linux receipts contain raw one-second snapshots
+and CPU deltas ranked by busy cores, with process name, parent PID, birth ticks,
+RSS pages and unmatched boundary counts. They omit command arguments.
+
+Each row also reports `reapedChildCores`: CPU of children the process reaped
+during the interval, attributed to that parent. It shows which process ran
+commands too short-lived to appear in a snapshot, not the commands themselves,
+and can include CPU the children used before the interval started.
+
+Each interval reports `cpuBusyCores` per logical CPU. Busy CPU excludes idle,
+iowait and steal time. A CPU brought online or offline during the interval
+reports `null`.
+The collector uses its own `/proc/self/status` `VmHWM` for peak RSS;
+`ru_maxrss` can retain a launcher's inherited high-water mark.
+It records observer CPU and scan duration. Report those costs beside the results; no
+negligible-impact claim follows from successful collection.
+
+On Windows, `scripts/process-attribution.ps1` accepts mandatory `OutputPath`
+and `StopPath` arguments. It saves process PID, start time, name and cumulative
+CPU, plus host activity and observer CPU/working set. Match PID and start time
+before computing deltas. It stops when the stop file exists or after 120 samples;
+choose a fresh stop path. Use the host's approved PowerShell invocation without
+changing execution policy.
+
+Neither collector stops jobs or changes any setting. Boundary samples
+miss processes that start and exit between them; unreadable processes are counted
+as missing. CPU contributors explain observed activity, not whether it changed
+application latency.
+
+## Scheduling controls
+
+These Linux controls change resource allocation under competition; they do not
+establish isolation. Check support for the actual kernel, controller and I/O
+path. Record effective values and unavailable controls for every arm; requesting
+a setting does not prove it took effect.
+
+| Control                                                                          | Effect and limit                                                                                                                                                                    |
+| -------------------------------------------------------------------------------- | ----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- |
+| [Nice level](https://man7.org/linux/man-pages/man2/nice.2.html)                  | Lower numeric values raise fair CPU scheduling priority. Permissions and autogroup policy affect the result. This does not reserve a core.                                          |
+| CPU affinity (`taskset`)                                                         | Restricts available CPUs and can affect worker counts and parallelism. It does not reserve exclusive cores. Use it for a declared CPU configuration, not automatic noise reduction. |
+| [CPU and I/O cgroup weights](https://docs.kernel.org/admin-guide/cgroup-v2.html) | Adjust relative shares among active sibling groups. Parent limits and controller/device support still apply; a higher weight is not a capacity guarantee.                           |
+| [Process I/O priority](https://man7.org/linux/man-pages/man2/ioprio_set.2.html)  | Depends on the supporting I/O scheduler and path. Process-specific priority does not cover asynchronous writes.                                                                     |
+| [Memory protection](https://docs.kernel.org/admin-guide/cgroup-v2.html)          | `memory.low` and `memory.min` protect against reclaim, not memory-bandwidth competition. Excessive hard protection can cause OOM. Memory caps remain limits, not reservations.      |
+
+In a virtual machine, guest affinity does not reserve host cores; host
+activity, shared cache, memory bandwidth and power limits still apply.
+
+Avoid realtime [CPU](https://man7.org/linux/man-pages/man7/sched.7.html) or
+[I/O](https://man7.org/linux/man-pages/man1/ionice.1.html) classes for generic
+benchmark noise reduction; they can starve dependencies or monitoring.
+Preserve the host's scheduling policy unless it authorizes a different regime.
+
+## Profiling and debugging
+
+Use a separate diagnostic run; instrumentation changes the work it observes.
+
+| Question                                    | Tool and reason                                                                                                                                                                                                                                       | Limits to record                                                                                                                          |
+| ------------------------------------------- | ----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | ----------------------------------------------------------------------------------------------------------------------------------------- |
+| Where does JavaScript CPU time go?          | Runtime CPU profiler: [Node](https://nodejs.org/api/cli.html#--cpu-prof) or [Bun](https://bun.com/docs/project/benchmarking#cpu-profiling).                                                                                                           | Match runtime/version and profile settings. Samples locate hotspots; verify improvements with unprofiled runs.                            |
+| Which JS objects are allocated or retained? | Runtime allocation profiles and heap snapshots: [Node](https://nodejs.org/api/cli.html#--heap-prof), [Bun](https://bun.com/docs/project/benchmarking#heap-profiling).                                                                                 | Matching flags do not guarantee matching metrics; see the distinction below.                                                              |
+| Where does native/kernel CPU work go?       | Linux [perf stat](https://github.com/torvalds/linux/blob/master/tools/perf/Documentation/perf-stat.txt) and [perf record](https://github.com/torvalds/linux/blob/master/tools/perf/Documentation/perf-record.txt): counters, samples and call graphs. | Permissions, hardware/kernel support, symbols and stack unwinding affect evidence. Frame-pointer unwinding needs compatible binaries.     |
+| Which system calls fail or wait?            | Linux [strace](https://github.com/strace/strace/blob/master/doc/strace.1.in): syscall results, errors and ordering.                                                                                                                                   | Interception changes execution; summaries are not ordinary throughput. Traces may contain sensitive paths or payloads.                    |
+| Which native allocations churn?             | Linux [heaptrack](https://github.com/KDE/heaptrack): allocation counts, temporary allocations and allocation stacks.                                                                                                                                  | Symbols improve attribution. Pool allocators need annotations; native allocation traces do not identify individual GC-managed JS objects. |
+
+### Native and JavaScript CPU profiles
+
+Keep native symbols in the diagnostic build. For a Rust release build, use
+`CARGO_PROFILE_RELEASE_DEBUG=line-tables-only CARGO_PROFILE_RELEASE_STRIP=none`.
+For Node JIT frames under `perf`, add `--perf-basic-prof`. A runtime CPU profile
+may attribute a native addon's work to its JavaScript caller; use native samples
+to see inside that call. Node `--cpu-prof` writes separate files for worker threads;
+retain them alongside the main-thread profile.
+
+If `perf report` stalls while expanding a large binary's inline frames, try
+`perf report --stdio --no-inline`. This omits inline-call expansion, so record
+that limit when attributing samples.
+
+### Viewing profiles
+
+[speedscope](https://github.com/jlfwong/speedscope) opens profiles locally as
+interactive flame graphs, including Node CPU profiles and Linux `perf` captures.
+It has been used to inspect JavaScript and native stacks across worker threads.
+To import a `perf` capture, write `perf script > profile.perf` without a custom
+`-F` field list; a partially recognized format can display one stack without an
+import error. Check that the expected threads and stacks appear.
+
+The npm CLI opens the default browser with `speedscope <profile>`. When using a
+controlled browser, serve its packaged `dist/release/` directory and the profile
+over loopback HTTP, then open `index.html#profileURL=<URL-encoded-profile-url>`.
+Use `encodeURIComponent(profileUrl)` for the value so query parameters remain
+part of the profile URL. Choose a free port and keep private captures on the
+local host. Follow the host's browser rules.
+
+### Allocation and waiting-time limits
+
+`heaptrack` unwinds a stack on each native allocation. Start with a small input:
+millions of allocations can make instrumentation dominate runtime. Treat that
+runtime as diagnostic overhead, and verify improvements with uninstrumented runs.
+
+On-CPU profiles omit time spent off CPU. Use a separate
+[bpftrace](https://bpftrace.org/docs) scheduler trace when a thread's wall time
+greatly exceeds its CPU time. Track the target and its descendants, and aggregate
+time between switch-out and switch-in. Off-CPU time includes both blocked time
+and time runnable threads wait for a CPU; sums across threads can exceed wall time.
+Kernel tracepoints, PID namespaces and privileges affect attribution, so verify
+the target's identity and that probes attached before starting it.
+
+Node `--heap-prof` samples allocation activity. Bun's documentation describes
+a full V8-format heap snapshot at exit, not the same sampled allocation history;
+its accepted interval flag does not add allocation sampling. Record which kind
+of artifact was captured and verify semantics for the version used.
+
+For a concrete crash, hang or event-tracing question, use the available
+[debugger](https://www.sourceware.org/gdb/documentation/) or
+[system tracing tool](https://bpftrace.org/docs). They are optional diagnostic
+tools, not benchmark prerequisites; platform and permissions affect availability.
+
+For TypeScript compiler versus application timing, and for Bun-versus-Node pitfalls, read [the language reference](javascript.md).
+
+## Correctness and audit evidence
+
+Use the repository's lint, type checks, focused tests and relevant static or
+security analysis. These establish intended behavior and detect defects; passing
+them does not establish speed or memory use. State their coverage limits.
+
+Add a tool after using it successfully on a real task. Record the question it
+answered, why it helped, its platform and measurement limits, and the official
+documentation. Keep untried candidates in research notes until then.