@bompus/house-rules 0.11.0 → 0.13.0
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- package/CODE_OF_CONDUCT.md +83 -0
- package/README.md +91 -60
- package/compose.mjs +8 -2
- package/composition.mjs +11 -3
- package/config-view.mjs +18 -3
- package/config.mjs +74 -13
- package/docs/assets/overview.png +0 -0
- package/docs/configuration.md +85 -9
- package/package.json +4 -2
- package/rules/core.md +118 -209
- package/rules/modifiers/backlog-outline.md +25 -0
- package/rules/modifiers/backlog-visual.md +16 -0
- package/rules/modifiers/coded-offers.md +47 -17
- package/rules/modifiers/cross-session-relay.md +23 -0
- package/rules/modifiers/low-quota-handoff.md +4 -3
- package/rules/modifiers/model-default-questions.md +33 -0
- package/rules/modifiers/multi-agent.md +6 -2
- package/rules/modifiers/no-attribution.md +7 -1
- package/rules/modifiers/periodic-updates.md +25 -0
- package/rules/modifiers/plan-files.md +35 -3
- package/rules/modifiers/release-batching.md +4 -2
- package/rules/modifiers/shared-host-load.md +13 -42
- package/rules/modifiers/squash-landing.md +1 -1
- package/rules/modifiers/upstream-contributions.md +45 -0
- package/rules/modifiers/web-image-formats.md +35 -0
- package/rules/references/backlog-view.md +154 -0
- package/rules/references/landing.md +14 -14
- package/rules/references/multi-agent.md +6 -2
- package/rules/references/release-batching.md +27 -4
- package/rules/references/squash-landing.md +10 -4
- package/rules/references/thread-visuals.md +45 -0
- package/setup.mjs +87 -17
- package/skills/hr-agent-guidance-audit/SKILL.md +10 -5
- package/skills/hr-agent-guidance-audit/references/prune-workflow.md +72 -0
- package/skills/hr-agent-guidance-audit/references/records-and-lessons.md +1 -1
- package/skills/hr-agent-guidance-audit/references/retirement-verdicts.md +49 -0
- package/skills/hr-agent-guidance-refresh/SKILL.md +25 -57
- package/skills/hr-agent-guidance-refresh/references/verification.md +1 -1
- package/skills/hr-audit-choices/SKILL.md +1 -1
- package/skills/hr-benchmarking/SKILL.md +122 -159
- package/skills/hr-benchmarking/references/javascript.md +113 -0
- package/skills/hr-benchmarking/references/tools.md +62 -57
- package/skills/hr-better-accessibility/SKILL.md +1 -1
- package/skills/hr-better-accessibility/screen-readers.md +1 -0
- package/skills/hr-code-review/SKILL.md +19 -3
- package/skills/hr-code-review/agents/openai.yaml +1 -1
- package/skills/hr-code-review/references/acp-client.md +43 -0
- package/skills/{hr-change-impact/SKILL.md → hr-code-review/references/impact.md} +2 -12
- package/skills/hr-code-review/references/panel-adjudication.md +58 -0
- package/skills/hr-code-review/references/panel-plan.md +61 -0
- package/skills/hr-code-review/references/panel-recommendations.json +105 -0
- package/skills/hr-code-review/references/panel-recommendations.md +46 -0
- package/skills/hr-code-review/references/panel-setup.md +54 -0
- package/skills/hr-code-review/references/reviewer-role.md +31 -0
- package/skills/hr-code-review/scripts/acp-panel-client.mjs +299 -0
- package/skills/hr-code-review/scripts/check-receipts.mjs +263 -0
- package/skills/hr-code-review/scripts/panel-plan.mjs +69 -0
- package/skills/hr-code-review/scripts/panel-preferences.mjs +220 -0
- package/skills/hr-design-exploration/SKILL.md +24 -1
- package/skills/hr-design-exploration/agents/openai.yaml +2 -2
- package/skills/hr-diagnosing-bugs/SKILL.md +2 -2
- package/skills/hr-diagnosing-bugs/references/reporting.md +29 -4
- package/skills/hr-diagnosing-bugs/scripts/check-pr-offer.mjs +47 -0
- package/skills/hr-explain-code/SKILL.md +1 -1
- package/skills/hr-extract-shared-steps/SKILL.md +1 -1
- package/skills/hr-forgotten-work-audit/SKILL.md +130 -0
- package/skills/hr-forgotten-work-audit/references/brief-template.md +56 -0
- package/skills/hr-forgotten-work-audit/references/result-format.md +46 -0
- package/skills/hr-handoff/LICENSE +21 -0
- package/skills/hr-handoff/NOTICE.md +29 -0
- package/skills/hr-handoff/SKILL.md +24 -1
- package/skills/hr-handoff/references/reconciliation.md +25 -6
- package/skills/hr-handoff/references/successor-thread.md +76 -0
- package/skills/hr-handoff/scripts/check-reconciliation.mjs +48 -6
- package/skills/hr-house-rules-setup/SKILL.md +22 -8
- package/skills/hr-lean-plan/SKILL.md +4 -0
- package/skills/hr-maintainability-review/SKILL.md +14 -4
- package/skills/hr-maintainability-review/agents/openai.yaml +1 -1
- package/skills/hr-manual-qa/SKILL.md +1 -1
- package/skills/hr-ordering-tests/SKILL.md +1 -1
- package/skills/hr-plain-prose/SKILL.md +7 -4
- package/skills/hr-pr-followup/SKILL.md +6 -3
- package/skills/hr-pr-followup/references/review-helpers.md +21 -0
- package/skills/hr-pr-followup/scripts/check-landing.py +204 -0
- package/skills/hr-pr-followup/scripts/merge-pr.sh +100 -0
- package/skills/hr-pr-followup/scripts/verify-landed.sh +184 -0
- package/skills/hr-read-reddit/SKILL.md +1 -1
- package/skills/hr-research-shared-links/SKILL.md +103 -0
- package/skills/hr-sqlite/SKILL.md +174 -0
- package/skills/hr-sqlite/agents/openai.yaml +2 -0
- package/skills/hr-sqlite/references/evidence.md +94 -0
- package/skills/hr-sqlite/references/tool-decisions.md +39 -0
- package/skills/hr-sqlite/scripts/inspect-sqlite.py +245 -0
- package/skills/hr-stock-ui-audit/SKILL.md +1 -1
- package/skills/hr-test-audit/LICENSE +21 -0
- package/skills/hr-test-audit/NOTICE.md +67 -0
- package/skills/hr-usage-report/SKILL.md +90 -0
- package/skills/hr-usage-report/agents/openai.yaml +2 -0
- package/skills/hr-usage-report/assets/example-report.json +125 -0
- package/skills/hr-usage-report/assets/report-template.html +377 -0
- package/skills/hr-usage-report/scripts/render-report.ts +86 -0
- package/skills/hr-usage-report/scripts/usage-by-model.ts +499 -0
- package/skills/hr-what-next/SKILL.md +100 -10
- package/skills/hr-what-next/references/reporting.md +13 -22
- package/skills/hr-what-next/scripts/check-open-work.mjs +711 -0
- package/skills/hr-writing-pr/SKILL.md +29 -14
- package/skills/hr-writing-pr/scripts/benefit-chart.py +200 -0
- package/docs/npm-release.md +0 -45
- package/rules/modifiers/swarmail.md +0 -12
- package/skills/hr-benchmarking/references/priority-findings-20261006.md +0 -115
- package/skills/hr-change-impact/agents/openai.yaml +0 -5
|
@@ -10,7 +10,7 @@ relying on flags or interpreting output.
|
|
|
10
10
|
| ---------------------------------- | ------------------------------------------------------------------------------------------------------------------------------------------------------ | -------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- |
|
|
11
11
|
| Whole CLI commands | [Hyperfine](https://github.com/sharkdp/hyperfine): repeated runs with wall time, CPU time and peak memory, warmup/preparation and JSON output. | Works across Linux, macOS and Windows. Declare warm/cold conditions. Since 2.0 commands run without a shell unless `-S` or `--shell` is given; 1.x needs `-N` for that. Peak memory and `--metrics` hardware counters are measured on Linux and macOS only; on Windows use another tool for memory. Check `schema_version` before reading exported JSON; 2.0 changed its layout. |
|
|
12
12
|
| JavaScript operations | [Tinybench](https://github.com/tinylibs/tinybench), directly or through the project's benchmark runner: warmup and task latency/throughput statistics. | The unit is a task invocation. For 1,000-operation batches, batch p99 divided by 1,000 is the p99 of batch-average costs, not mean operation cost or individual-operation p99. |
|
|
13
|
-
| HTTP load, including JS/TS servers | [oha](https://github.com/hatoo/oha): a native load generator with concurrency/rate controls, latency distributions and JSON output. | Declare protocol, connections, rate, deadlines, keepalive and error handling. Check generator CPU/network limits.
|
|
13
|
+
| HTTP load, including JS/TS servers | [oha](https://github.com/hatoo/oha): a native load generator with concurrency/rate controls, latency distributions and JSON output. | Declare protocol, connections, rate, deadlines, keepalive and error handling. Check generator CPU/network limits. For fixed-rate load, `--latency-correction` corrects coordinated omission; it is ignored unless `-q` is set. |
|
|
14
14
|
| Command CPU and memory | [GNU time](https://www.gnu.org/software/time/manual/time.html), commonly `/usr/bin/time -v`: wall time, CPU and maximum RSS. | GNU options differ from shell `time` and BSD/macOS tools. RSS is a lifetime high-water mark for the observed process; it does not isolate retained cache memory. |
|
|
15
15
|
| Host or process contention | Linux [mpstat/pidstat](https://github.com/sysstat/sysstat): processor utilization and process CPU, memory and I/O observations. | Sampling can miss short bursts; some counters need kernel support. These explain interference, not application latency. |
|
|
16
16
|
|
|
@@ -44,14 +44,13 @@ during the interval, attributed to that parent. It shows which process ran
|
|
|
44
44
|
commands too short-lived to appear in a snapshot, not the commands themselves,
|
|
45
45
|
and can include CPU the children used before the interval started.
|
|
46
46
|
|
|
47
|
-
Each interval reports `cpuBusyCores` per logical CPU
|
|
48
|
-
|
|
49
|
-
|
|
50
|
-
fenced-core gate rejects that interval.
|
|
47
|
+
Each interval reports `cpuBusyCores` per logical CPU. Busy CPU excludes idle,
|
|
48
|
+
iowait and steal time. A CPU brought online or offline during the interval
|
|
49
|
+
reports `null`.
|
|
51
50
|
The collector uses its own `/proc/self/status` `VmHWM` for peak RSS;
|
|
52
51
|
`ru_maxrss` can retain a launcher's inherited high-water mark.
|
|
53
|
-
It records observer CPU and scan duration.
|
|
54
|
-
|
|
52
|
+
It records observer CPU and scan duration. Report those costs beside the results; no
|
|
53
|
+
negligible-impact claim follows from successful collection.
|
|
55
54
|
|
|
56
55
|
On Windows, `scripts/process-attribution.ps1` accepts mandatory `OutputPath`
|
|
57
56
|
and `StopPath` arguments. It saves process PID, start time, name and cumulative
|
|
@@ -60,10 +59,10 @@ before computing deltas. It stops when the stop file exists or after 120 samples
|
|
|
60
59
|
choose a fresh stop path. Use the host's approved PowerShell invocation without
|
|
61
60
|
changing execution policy.
|
|
62
61
|
|
|
63
|
-
Neither collector stops jobs
|
|
62
|
+
Neither collector stops jobs or changes any setting. Boundary samples
|
|
64
63
|
miss processes that start and exit between them; unreadable processes are counted
|
|
65
64
|
as missing. CPU contributors explain observed activity, not whether it changed
|
|
66
|
-
application latency.
|
|
65
|
+
application latency.
|
|
67
66
|
|
|
68
67
|
## Scheduling controls
|
|
69
68
|
|
|
@@ -72,53 +71,16 @@ establish isolation. Check support for the actual kernel, controller and I/O
|
|
|
72
71
|
path. Record effective values and unavailable controls for every arm; requesting
|
|
73
72
|
a setting does not prove it took effect.
|
|
74
73
|
|
|
75
|
-
| Control | Effect and limit
|
|
76
|
-
| -------------------------------------------------------------------------------- |
|
|
77
|
-
| [Nice level](https://man7.org/linux/man-pages/man2/nice.2.html) | Lower numeric values raise fair CPU scheduling priority. Permissions and autogroup policy affect the result. This does not reserve a core.
|
|
78
|
-
| CPU affinity (`taskset`) | Restricts available CPUs and can affect worker counts and parallelism. It does not reserve exclusive cores. Use it for a declared CPU configuration
|
|
79
|
-
| [CPU and I/O cgroup weights](https://docs.kernel.org/admin-guide/cgroup-v2.html) | Adjust relative shares among active sibling groups. Parent limits and controller/device support still apply; a higher weight is not a capacity guarantee.
|
|
80
|
-
| [Process I/O priority](https://man7.org/linux/man-pages/man2/ioprio_set.2.html) | Depends on the supporting I/O scheduler and path. Process-specific priority does not cover asynchronous writes.
|
|
81
|
-
| [Memory protection](https://docs.kernel.org/admin-guide/cgroup-v2.html) | `memory.low` and `memory.min` protect against reclaim, not memory-bandwidth competition. Excessive hard protection can cause OOM. Memory caps remain limits, not reservations.
|
|
82
|
-
|
|
83
|
-
|
|
84
|
-
|
|
85
|
-
On a shared host where brief bursts from other sessions keep failing host-wide
|
|
86
|
-
admission, a fenced-core arm keeps other work off chosen CPUs for a finite phase. It is a
|
|
87
|
-
declared sensitivity arm that needs the user's authorization, because it changes
|
|
88
|
-
other sessions' CPU placement:
|
|
89
|
-
|
|
90
|
-
1. Choose the benchmark CPUs as whole physical cores (include SMT siblings) and pin
|
|
91
|
-
every arm, its browser or runtime and its children to them.
|
|
92
|
-
2. Move every other thread you are permitted to change off the benchmark CPUs.
|
|
93
|
-
Record each thread's identity (PID, thread ID, birth ticks) and original
|
|
94
|
-
affinity. Stop and signal nothing. A thread that sets its own affinity, or
|
|
95
|
-
is created with an explicit mask, can stay eligible on the benchmark CPUs;
|
|
96
|
-
the gate still counts its CPU.
|
|
97
|
-
3. Gate on foreign CPU on the benchmark CPUs: their busy time (`cpuBusyCores`)
|
|
98
|
-
minus the interval CPU of the benchmark's own live processes (`busyCores`).
|
|
99
|
-
Do not subtract `reapedChildCores`: it can include CPU used before the
|
|
100
|
-
interval, so it would understate foreign load. Process CPU is not per-CPU,
|
|
101
|
-
so the subtraction holds only while the whole tree stays on the benchmark
|
|
102
|
-
CPUs: check every thread's effective affinity at the start and end of each
|
|
103
|
-
batch, and reject the batch if any ran elsewhere. Boundary checks miss a
|
|
104
|
-
thread that widens and restores its affinity within the batch; when the
|
|
105
|
-
workload can change its own affinity, confine the tree with a cpuset it
|
|
106
|
-
cannot widen. In the same census, record every other permitted thread
|
|
107
|
-
still eligible on the benchmark CPUs as unfenced. Report host-wide load as
|
|
108
|
-
context.
|
|
109
|
-
4. At the phase boundary, restore recorded threads after checking their birth
|
|
110
|
-
ticks. Threads started during the window inherited the fence; give them the
|
|
111
|
-
CPU set they would otherwise have inherited, usually all CPUs. A still-fenced
|
|
112
|
-
thread can start another while you restore, so repeat the census until a
|
|
113
|
-
full pass finds no permitted thread left fenced; threads started after their
|
|
114
|
-
creator is restored inherit the restored set. Do not freeze processes to stop
|
|
115
|
-
thread creation. Run this cleanup even when the phase fails.
|
|
116
|
-
|
|
117
|
-
Report results as constrained to the pinned setting with the effective
|
|
118
|
-
affinities. Processes you cannot change (other users, system services) stay
|
|
119
|
-
unfenced; record them. In a virtual machine, guest affinity does not reserve
|
|
120
|
-
host cores; host activity, shared cache, memory bandwidth and power limits
|
|
121
|
-
still apply.
|
|
74
|
+
| Control | Effect and limit |
|
|
75
|
+
| -------------------------------------------------------------------------------- | ----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- |
|
|
76
|
+
| [Nice level](https://man7.org/linux/man-pages/man2/nice.2.html) | Lower numeric values raise fair CPU scheduling priority. Permissions and autogroup policy affect the result. This does not reserve a core. |
|
|
77
|
+
| CPU affinity (`taskset`) | Restricts available CPUs and can affect worker counts and parallelism. It does not reserve exclusive cores. Use it for a declared CPU configuration, not automatic noise reduction. |
|
|
78
|
+
| [CPU and I/O cgroup weights](https://docs.kernel.org/admin-guide/cgroup-v2.html) | Adjust relative shares among active sibling groups. Parent limits and controller/device support still apply; a higher weight is not a capacity guarantee. |
|
|
79
|
+
| [Process I/O priority](https://man7.org/linux/man-pages/man2/ioprio_set.2.html) | Depends on the supporting I/O scheduler and path. Process-specific priority does not cover asynchronous writes. |
|
|
80
|
+
| [Memory protection](https://docs.kernel.org/admin-guide/cgroup-v2.html) | `memory.low` and `memory.min` protect against reclaim, not memory-bandwidth competition. Excessive hard protection can cause OOM. Memory caps remain limits, not reservations. |
|
|
81
|
+
|
|
82
|
+
In a virtual machine, guest affinity does not reserve host cores; host
|
|
83
|
+
activity, shared cache, memory bandwidth and power limits still apply.
|
|
122
84
|
|
|
123
85
|
Avoid realtime [CPU](https://man7.org/linux/man-pages/man7/sched.7.html) or
|
|
124
86
|
[I/O](https://man7.org/linux/man-pages/man1/ionice.1.html) classes for generic
|
|
@@ -137,6 +99,49 @@ Use a separate diagnostic run; instrumentation changes the work it observes.
|
|
|
137
99
|
| Which system calls fail or wait? | Linux [strace](https://github.com/strace/strace/blob/master/doc/strace.1.in): syscall results, errors and ordering. | Interception changes execution; summaries are not ordinary throughput. Traces may contain sensitive paths or payloads. |
|
|
138
100
|
| Which native allocations churn? | Linux [heaptrack](https://github.com/KDE/heaptrack): allocation counts, temporary allocations and allocation stacks. | Symbols improve attribution. Pool allocators need annotations; native allocation traces do not identify individual GC-managed JS objects. |
|
|
139
101
|
|
|
102
|
+
### Native and JavaScript CPU profiles
|
|
103
|
+
|
|
104
|
+
Keep native symbols in the diagnostic build. For a Rust release build, use
|
|
105
|
+
`CARGO_PROFILE_RELEASE_DEBUG=line-tables-only CARGO_PROFILE_RELEASE_STRIP=none`.
|
|
106
|
+
For Node JIT frames under `perf`, add `--perf-basic-prof`. A runtime CPU profile
|
|
107
|
+
may attribute a native addon's work to its JavaScript caller; use native samples
|
|
108
|
+
to see inside that call. Node `--cpu-prof` writes separate files for worker threads;
|
|
109
|
+
retain them alongside the main-thread profile.
|
|
110
|
+
|
|
111
|
+
If `perf report` stalls while expanding a large binary's inline frames, try
|
|
112
|
+
`perf report --stdio --no-inline`. This omits inline-call expansion, so record
|
|
113
|
+
that limit when attributing samples.
|
|
114
|
+
|
|
115
|
+
### Viewing profiles
|
|
116
|
+
|
|
117
|
+
[speedscope](https://github.com/jlfwong/speedscope) opens profiles locally as
|
|
118
|
+
interactive flame graphs, including Node CPU profiles and Linux `perf` captures.
|
|
119
|
+
It has been used to inspect JavaScript and native stacks across worker threads.
|
|
120
|
+
To import a `perf` capture, write `perf script > profile.perf` without a custom
|
|
121
|
+
`-F` field list; a partially recognized format can display one stack without an
|
|
122
|
+
import error. Check that the expected threads and stacks appear.
|
|
123
|
+
|
|
124
|
+
The npm CLI opens the default browser with `speedscope <profile>`. When using a
|
|
125
|
+
controlled browser, serve its packaged `dist/release/` directory and the profile
|
|
126
|
+
over loopback HTTP, then open `index.html#profileURL=<URL-encoded-profile-url>`.
|
|
127
|
+
Use `encodeURIComponent(profileUrl)` for the value so query parameters remain
|
|
128
|
+
part of the profile URL. Choose a free port and keep private captures on the
|
|
129
|
+
local host. Follow the host's browser rules.
|
|
130
|
+
|
|
131
|
+
### Allocation and waiting-time limits
|
|
132
|
+
|
|
133
|
+
`heaptrack` unwinds a stack on each native allocation. Start with a small input:
|
|
134
|
+
millions of allocations can make instrumentation dominate runtime. Treat that
|
|
135
|
+
runtime as diagnostic overhead, and verify improvements with uninstrumented runs.
|
|
136
|
+
|
|
137
|
+
On-CPU profiles omit time spent off CPU. Use a separate
|
|
138
|
+
[bpftrace](https://bpftrace.org/docs) scheduler trace when a thread's wall time
|
|
139
|
+
greatly exceeds its CPU time. Track the target and its descendants, and aggregate
|
|
140
|
+
time between switch-out and switch-in. Off-CPU time includes both blocked time
|
|
141
|
+
and time runnable threads wait for a CPU; sums across threads can exceed wall time.
|
|
142
|
+
Kernel tracepoints, PID namespaces and privileges affect attribution, so verify
|
|
143
|
+
the target's identity and that probes attached before starting it.
|
|
144
|
+
|
|
140
145
|
Node `--heap-prof` samples allocation activity. Bun's documentation describes
|
|
141
146
|
a full V8-format heap snapshot at exit, not the same sampled allocation history;
|
|
142
147
|
its accepted interval flag does not add allocation sampling. Record which kind
|
|
@@ -147,7 +152,7 @@ For a concrete crash, hang or event-tracing question, use the available
|
|
|
147
152
|
[system tracing tool](https://bpftrace.org/docs). They are optional diagnostic
|
|
148
153
|
tools, not benchmark prerequisites; platform and permissions affect availability.
|
|
149
154
|
|
|
150
|
-
For TypeScript compiler versus application timing, read [the language reference](javascript.md).
|
|
155
|
+
For TypeScript compiler versus application timing, and for Bun-versus-Node pitfalls, read [the language reference](javascript.md).
|
|
151
156
|
|
|
152
157
|
## Correctness and audit evidence
|
|
153
158
|
|
|
@@ -7,7 +7,7 @@ metadata:
|
|
|
7
7
|
upstream_ref: 267330e1adfc66a718fb65fa6918c1f06d0a689e
|
|
8
8
|
upstream_path: skills/better-accessibility/
|
|
9
9
|
local_patches: "Use-when description; reduced-motion kill switch spares spinners; 320px reflow keeps the 2D-content exception; carousel pan-y, touch-callout and real-hardware check (from emilkowalski/skills mobile-native)"
|
|
10
|
-
house_rules_patches: "Portable provenance and review verdict"
|
|
10
|
+
house_rules_patches: "Portable provenance and review verdict; unique SVG title and desc ids"
|
|
11
11
|
mobile_native_source: https://github.com/emilkowalski/skills/blob/e8a175de22ae1e49370fc144c1f3bb9aeedf988d/skills/mobile-native/SKILL.md
|
|
12
12
|
---
|
|
13
13
|
|
|
@@ -84,6 +84,7 @@ A missing `alt` is worse than an empty one, because screen readers fall back to
|
|
|
84
84
|
|
|
85
85
|
- Decorative SVG: `aria-hidden="true"` and `focusable="false"`, the latter for legacy Edge and IE tabbing. No title needed.
|
|
86
86
|
- Meaningful inline SVG: `role="img"` plus `aria-label="…"`, or a `<title>` as the first child referenced by `aria-labelledby`.
|
|
87
|
+
- Give each `<title>` and `<desc>` an `id` that is unique on the page, such as `chart-revenue-title`. Two inline SVGs that share a bare `id="title"` both resolve to the first one. Keep `<title>` to a short name and make `<desc>` one sentence about what the figure shows, not how it is drawn.
|
|
87
88
|
- Simple cases: `<img src="icon.svg" alt="…">` is the most reliable delivery.
|
|
88
89
|
|
|
89
90
|
```tsx
|
|
@@ -1,6 +1,6 @@
|
|
|
1
1
|
---
|
|
2
2
|
name: hr-code-review
|
|
3
|
-
description: Review a branch, PR, working-tree changes or changes since a revision for repository standards and
|
|
3
|
+
description: Review a branch, PR, working-tree changes or changes since a revision for repository standards, spec conformance and what the change can break beyond its diff. Use when asked for a change review, what a change might break before merging, or to set up or change a review panel or for panel recommendations. Not for simplification-only or strictest maintainability audits.
|
|
4
4
|
---
|
|
5
5
|
|
|
6
6
|
Two-axis review of the changes the user requested: working-tree changes, a committed branch, or changes since a named revision.
|
|
@@ -8,10 +8,22 @@ Two-axis review of the changes the user requested: working-tree changes, a commi
|
|
|
8
8
|
- **Standards**: does the code conform to this repo's documented coding standards?
|
|
9
9
|
- **Spec**: does the code implement the originating issue / spec?
|
|
10
10
|
|
|
11
|
-
Keep both axes distinct.
|
|
11
|
+
Keep both axes distinct. Add an **Impact** section when the user asks what the
|
|
12
|
+
change might break, or when the diff changes something read outside its own
|
|
13
|
+
lines: a stored or serialized shape, output another process consumes, timing,
|
|
14
|
+
configuration, or reliance on pinned dependency behavior. Follow
|
|
15
|
+
[the change-impact method](references/impact.md) for it. Review small scopes locally; use separate read-only
|
|
12
16
|
subagents when the scope benefits from independent review and the host permits
|
|
13
17
|
it. Missing spec material limits the Spec assessment, not all useful review.
|
|
14
18
|
|
|
19
|
+
When a selected review uses an ACP stdio agent, read [the bundled client contract](references/acp-client.md) before launching it. Host orchestration and authorization still determine the route.
|
|
20
|
+
|
|
21
|
+
When a panel uses stored preferences, read [the panel plan contract](references/panel-plan.md) to resolve the seats before launching any of them.
|
|
22
|
+
|
|
23
|
+
When a panel has answered and you must combine its replies, read [the panel adjudication procedure](references/panel-adjudication.md); it includes a citation gate.
|
|
24
|
+
|
|
25
|
+
When the user asks to set up or change a panel, for panel recommendations, or to use a panel for the first time without naming every seat, read [the panel setup procedure](references/panel-setup.md); it loads [the recommendations](references/panel-recommendations.md) when needed.
|
|
26
|
+
|
|
15
27
|
## Process
|
|
16
28
|
|
|
17
29
|
### 1. Pin the review scope
|
|
@@ -70,7 +82,7 @@ results and verification limits.
|
|
|
70
82
|
When delegating and both sources are available, the reviews may run in parallel.
|
|
71
83
|
If a source arrives later or scope changes, update only the affected assessment.
|
|
72
84
|
|
|
73
|
-
**Hosts.** Both sub-agents are read-only. Use the current host's native agent mechanism and follow the repository's model-selection rules. A sub-agent never runs the model that wrote the code; when the author is its vendor's strongest model, use another vendor's model where one is available, through the host or that vendor's command-line agent run read-only with the same brief. When no other model is available, use a fresh context and say in the report that the reviewer shares the author's model.
|
|
85
|
+
**Hosts.** A standing reviewer or a read-only sub-agent given one review uses [the independent review role](references/reviewer-role.md) and its PASS/BLOCK result. Both sub-agents are read-only. Use the current host's native agent mechanism and follow the repository's model-selection rules. A sub-agent never runs the model that wrote the code; when the author is its vendor's strongest model, use another vendor's model where one is available, through the host or that vendor's command-line agent run read-only with the same brief. When no other model is available, use a fresh context and say in the report that the reviewer shares the author's model.
|
|
74
86
|
|
|
75
87
|
**Correctness of input-to-output code** goes in the Spec brief, using the
|
|
76
88
|
evidence method above. Split a large scope into groups of about three files,
|
|
@@ -100,6 +112,10 @@ Present Standards and Spec separately. Verify delegated findings, remove
|
|
|
100
112
|
unsupported or duplicate claims, and prioritize within each axis. Preserve the
|
|
101
113
|
distinction between documented violations and heuristic suggestions.
|
|
102
114
|
|
|
115
|
+
Report Impact after the two axes: what the change relies on, its real risks and
|
|
116
|
+
what was ruled out, as the change-impact method describes. It never reranks
|
|
117
|
+
Standards or Spec findings.
|
|
118
|
+
|
|
103
119
|
Report a one-line summary: total findings per axis, and the worst issue _within each axis_ (if any). Don't pick a single winner across axes; the separation exists to prevent that reranking.
|
|
104
120
|
|
|
105
121
|
Tag each surviving finding HIGH (corroborated by a second source or a directly cited requirement), MEDIUM (single inspected source), or LOW (inferred; name the check that would confirm it). Nothing is dropped on confidence alone: the operator adjudicates.
|
|
@@ -0,0 +1,43 @@
|
|
|
1
|
+
# ACP review client
|
|
2
|
+
|
|
3
|
+
Use the bundled `../scripts/acp-panel-client.mjs` when the selected review needs
|
|
4
|
+
an ACP v1 agent over stdio. This helper does not choose a provider, model,
|
|
5
|
+
effort or review scope. It does not select a route based on whether it uses the usage
|
|
6
|
+
allowance. Launch it only within the user's authorization.
|
|
7
|
+
|
|
8
|
+
```sh
|
|
9
|
+
node <skill-directory>/scripts/acp-panel-client.mjs \
|
|
10
|
+
'["<agent-command>","<stdio-argument>"]' brief.md answer.txt <review-root> \
|
|
11
|
+
--model <advertised-model-id> --effort <advertised-effort-value> \
|
|
12
|
+
--verdict 'VERDICT:' --max-turns 2 --idle-min 5 --timeout-min 60
|
|
13
|
+
```
|
|
14
|
+
|
|
15
|
+
Node 22+ and Bun can run the same file. Arguments after the output path are
|
|
16
|
+
optional. The default is one turn, no required verdict, five minutes of idle
|
|
17
|
+
silence counted from launch and a sixty-minute hard deadline. The hard deadline has a one-minute
|
|
18
|
+
minimum. The caller supplies its environment and temporary-storage policy.
|
|
19
|
+
|
|
20
|
+
The client advertises confined file reads, disables writes and terminal
|
|
21
|
+
execution, and rejects all permission requests. Physical file targets must stay
|
|
22
|
+
inside the canonical review root. Put Git/GitHub evidence in the brief when the
|
|
23
|
+
agent cannot obtain it through its own approved route. Missing or mismatched
|
|
24
|
+
model/effort confirmations stop before the first prompt. Codex must advertise
|
|
25
|
+
and confirm `read-only` mode; any other advertised mode is also set to read-only.
|
|
26
|
+
|
|
27
|
+
These controls govern client-provided ACP capabilities. They do not sandbox an
|
|
28
|
+
agent's own tools, prevent dishonest setting reports, or eliminate filesystem
|
|
29
|
+
races during concurrent replacement. The caller must qualify the provider's
|
|
30
|
+
own read-only route. Git status comparison detects some workspace changes; it
|
|
31
|
+
does not detect ignored-file writes or prove that no writes occurred.
|
|
32
|
+
|
|
33
|
+
The output contains collected agent text. Exit codes are:
|
|
34
|
+
|
|
35
|
+
- 0: satisfied verdict condition
|
|
36
|
+
- 1: setup/protocol failure
|
|
37
|
+
- 2: missing arguments
|
|
38
|
+
- 3: incomplete verdict
|
|
39
|
+
- 4: changed Git status (the answer file is written after this comparison)
|
|
40
|
+
- 124: deadline, even when the agent honors the cancel
|
|
41
|
+
|
|
42
|
+
A permission-cancelled turn may continue within `--max-turns`. A matching marker
|
|
43
|
+
is a completion signal, not proof that the review is accurate.
|
|
@@ -1,11 +1,7 @@
|
|
|
1
|
-
---
|
|
2
|
-
name: hr-change-impact
|
|
3
|
-
description: "Check what a change can break beyond the lines it edits, and back the claim that it is safe with something that ran. Use when the user asks what a change might break, wants its impact checked before merging, or distrusts a small diff."
|
|
4
|
-
disable-model-invocation: true
|
|
5
|
-
---
|
|
6
|
-
|
|
7
1
|
# Change impact
|
|
8
2
|
|
|
3
|
+
Loaded by `hr-code-review` for its Impact section.
|
|
4
|
+
|
|
9
5
|
The diff shows what changed. This skill asks what else depends on the old
|
|
10
6
|
behavior, and whether the change is safe for a reason someone has seen work.
|
|
11
7
|
Follow project guidance for commands, test runners and domain safeguards.
|
|
@@ -105,9 +101,3 @@ Keep the list of risks short. Three real ones with locations beat fifteen
|
|
|
105
101
|
possibilities. Before sharing publicly, follow the project's data-classification
|
|
106
102
|
policy. Remove credentials, personal data, private project names and URLs,
|
|
107
103
|
and local paths.
|
|
108
|
-
|
|
109
|
-
## Related skills
|
|
110
|
-
|
|
111
|
-
- `hr-explain-code` when you need to understand the code before judging a change.
|
|
112
|
-
- `hr-code-review` for standards, style and spec conformance of the same diff.
|
|
113
|
-
- `hr-diagnosing-bugs` once something is actually broken.
|
|
@@ -0,0 +1,58 @@
|
|
|
1
|
+
# Adjudicate a panel
|
|
2
|
+
|
|
3
|
+
Use this after a panel of models has answered the same brief and you must turn
|
|
4
|
+
their replies into one result. It covers reading the replies, checking what they
|
|
5
|
+
cite and reporting the choices. Resolving the seats comes first and is in
|
|
6
|
+
[the panel plan contract](panel-plan.md).
|
|
7
|
+
|
|
8
|
+
## Read the replies
|
|
9
|
+
|
|
10
|
+
1. Extract each seat's text from its log. List what the seats agree on and where
|
|
11
|
+
they contradict each other.
|
|
12
|
+
2. For every claim that carries weight, note whether each seat cites a receipt
|
|
13
|
+
that passes the check below. Two seats that cite the same line, or neither
|
|
14
|
+
citing anything, are unverified agreement, not corroboration.
|
|
15
|
+
3. Run the citation gate on the logs:
|
|
16
|
+
|
|
17
|
+
```sh
|
|
18
|
+
node <skill-directory>/scripts/check-receipts.mjs --repo <repo> <panel-log>...
|
|
19
|
+
```
|
|
20
|
+
|
|
21
|
+
A citation written as `` `name` (path:line) `` passes only when the identifier
|
|
22
|
+
appears on the cited lines. A bare `path:line` passes when the file has that
|
|
23
|
+
line. A file named by basename or subpath must resolve to exactly one file in
|
|
24
|
+
the repository. Exit 1 means adjudicate those claims by hand; it does not mean
|
|
25
|
+
the panel failed. Exit 2 means no citations were found, a log could not be read or the
|
|
26
|
+
arguments were wrong; treat it as an unverified gate. Only exit 0 passes. A citation
|
|
27
|
+
to a file outside the repository fails.
|
|
28
|
+
|
|
29
|
+
4. A person checks the remaining claims against the source. Record the base you
|
|
30
|
+
chose, what you took from other seats, what you rejected and which seats
|
|
31
|
+
dropped out. Keep the logs in the task's scratch directory, not in the repository.
|
|
32
|
+
|
|
33
|
+
## Compare competing implementations
|
|
34
|
+
|
|
35
|
+
When the brief asks seats to produce competing implementations and a cheap
|
|
36
|
+
objective check (tests, a diff) exists, state the rubric before the run. The
|
|
37
|
+
result may name a base and graft the others' best parts by hand, so the whole
|
|
38
|
+
stays coherent. Name the premise the choice depends on and challenge it with one
|
|
39
|
+
concrete case, including any hand edits made after the verdict. A graft that fails
|
|
40
|
+
the case is recorded as unresolved, not named a winner.
|
|
41
|
+
|
|
42
|
+
## Break a tie
|
|
43
|
+
|
|
44
|
+
Give one seat both verdicts and your adjudication, and ask for residuals only,
|
|
45
|
+
under the same read-only contract. Residuals are what the reference method misses,
|
|
46
|
+
whether each surviving item is earned, and anything wrongly declined.
|
|
47
|
+
|
|
48
|
+
## Open the report with a receipt
|
|
49
|
+
|
|
50
|
+
Begin every panel report with this header so the choices can be audited without
|
|
51
|
+
re-reading the thread:
|
|
52
|
+
|
|
53
|
+
- **Seats:** each model, its family and its role (consensus, contrarian, tie-break).
|
|
54
|
+
- **Settings per seat:** the effort or variant actually used and the one-line rule
|
|
55
|
+
that selected it (task mapping, seat rule or the user's explicit choice).
|
|
56
|
+
- **Deviations:** any seat that did not run as briefed (fallback route, escalation,
|
|
57
|
+
dropout) and what replaced it.
|
|
58
|
+
- **Adjudication basis:** the gate result, what was checked by hand and what remains inference.
|
|
@@ -0,0 +1,61 @@
|
|
|
1
|
+
# Panel plan
|
|
2
|
+
|
|
3
|
+
Use the bundled `../scripts/panel-plan.mjs` to see which seats stored panel
|
|
4
|
+
preferences resolve to before any model runs. It reads files and prints a plan;
|
|
5
|
+
it never launches a seat, discovers a provider, checks quota or changes a file.
|
|
6
|
+
|
|
7
|
+
```sh
|
|
8
|
+
node <skill-directory>/scripts/panel-plan.mjs --bindings bindings.json \
|
|
9
|
+
--author-family <family-of-the-running-session> [--config <house-rules.json>] \
|
|
10
|
+
[--task task.json] [--project project.json --approve-project <sha256-of-project.json>] \
|
|
11
|
+
[--approve-metered <alias>] [--json]
|
|
12
|
+
```
|
|
13
|
+
|
|
14
|
+
Exit 0 means a plan, 1 means named blockers, 2 means unreadable or invalid input.
|
|
15
|
+
`--config` defaults to the house-rules configuration; its `panel` section is the
|
|
16
|
+
user layer. Every input is a regular file, not a symlink.
|
|
17
|
+
|
|
18
|
+
## Bindings
|
|
19
|
+
|
|
20
|
+
A user-owned file maps each alias to its exact launch and provenance. Only the
|
|
21
|
+
user's own file belongs here; credentials stay in the provider's authentication.
|
|
22
|
+
|
|
23
|
+
```json
|
|
24
|
+
{
|
|
25
|
+
"version": 1,
|
|
26
|
+
"aliases": {
|
|
27
|
+
"review-primary": {
|
|
28
|
+
"families": ["anthropic"],
|
|
29
|
+
"metered": false,
|
|
30
|
+
"launch": { "runner": "claude", "model": "<id>", "effort": "<level>" }
|
|
31
|
+
}
|
|
32
|
+
}
|
|
33
|
+
}
|
|
34
|
+
```
|
|
35
|
+
|
|
36
|
+
`launch` is opaque to the resolver: the caller interprets and validates it. An
|
|
37
|
+
omitted `families` means the family is unknown. An unknown family cannot satisfy
|
|
38
|
+
`distinctFamilies` or `excludeAuthorFamily`, and an unknown author family
|
|
39
|
+
cannot be excluded.
|
|
40
|
+
|
|
41
|
+
## Resolution
|
|
42
|
+
|
|
43
|
+
1. Layers apply in order user, project, task. A later layer replaces a role's
|
|
44
|
+
whole candidate list; its new roles append. Project preferences count only
|
|
45
|
+
with `--approve-project` equal to the file's SHA-256 revision; otherwise the
|
|
46
|
+
plan is blocked, not silently reduced to the user layer.
|
|
47
|
+
2. Policy only tightens: the lowest `roundLimit`, `fallback: none`,
|
|
48
|
+
`meteredRoutes: included-only`, and either layer's `distinctFamilies` or
|
|
49
|
+
`excludeAuthorFamily`. A reduced panel needs the user layer to allow it and no
|
|
50
|
+
other layer to forbid it.
|
|
51
|
+
3. A candidate is skipped, with a reason, when it has no binding, an unknown or
|
|
52
|
+
author family under strict policy, or a metered route (never allowed under
|
|
53
|
+
`included-only`; under `explicit-approval-required` only with
|
|
54
|
+
`--approve-metered <alias>`). With
|
|
55
|
+
`fallback: none` only the first candidate of each role is tried.
|
|
56
|
+
4. A role with no eligible candidate blocks the plan, or is dropped when a
|
|
57
|
+
reduced panel is allowed. Distinct families are found by trying later
|
|
58
|
+
candidates; if no combination works the plan is blocked.
|
|
59
|
+
|
|
60
|
+
The plan lists each seat, every skipped candidate and the round limit. It
|
|
61
|
+
certifies configuration only, not availability, quota or effective effort.
|
|
@@ -0,0 +1,105 @@
|
|
|
1
|
+
{
|
|
2
|
+
"version": 1,
|
|
3
|
+
"reviewed": "2026-10-14",
|
|
4
|
+
"entries": [
|
|
5
|
+
{
|
|
6
|
+
"id": "opus-5-5-bug-hunt",
|
|
7
|
+
"profiles": ["balanced"],
|
|
8
|
+
"task": "Find planted bugs: 105 across two repositories, count fixed",
|
|
9
|
+
"model": "claude-opus-5-5",
|
|
10
|
+
"harness": "Claude Code",
|
|
11
|
+
"route": "Anthropic first-party API",
|
|
12
|
+
"efforts": [
|
|
13
|
+
{
|
|
14
|
+
"effort": "medium",
|
|
15
|
+
"fixed": 30.3,
|
|
16
|
+
"listCostUsd": 15.68,
|
|
17
|
+
"wallMinutes": 17.0
|
|
18
|
+
},
|
|
19
|
+
{
|
|
20
|
+
"effort": "high",
|
|
21
|
+
"fixed": 31.7,
|
|
22
|
+
"listCostUsd": 22.25,
|
|
23
|
+
"wallMinutes": 23.9
|
|
24
|
+
},
|
|
25
|
+
{
|
|
26
|
+
"effort": "xhigh",
|
|
27
|
+
"fixed": 36.0,
|
|
28
|
+
"listCostUsd": 34.98,
|
|
29
|
+
"wallMinutes": 42.5
|
|
30
|
+
}
|
|
31
|
+
],
|
|
32
|
+
"sourceUrl": "https://bughunt.productcompass.pm/data/benchmark.json",
|
|
33
|
+
"checked": "2026-10-14",
|
|
34
|
+
"samples": "3 runs per effort, non-superseded rows; each run covers both repositories",
|
|
35
|
+
"metric": "planted bugs fixed, mean of 3, out of 105",
|
|
36
|
+
"timeDefinition": "wall clock summed over both repositories; the board says these runs shared the machine with other work",
|
|
37
|
+
"costDefinition": "list-rate API price estimated from usage counts, summed over both repositories; not an invoice or subscription billing",
|
|
38
|
+
"limitations": "Counts planted bugs, not review quality on real changes; repeated attempts are not independent replications; runs used a 1M context window, and the board says its older Opus 5 rows are not comparable to these; the board also lists low and max rungs that this entry leaves out",
|
|
39
|
+
"rationale": "Medium used about 30% less cost and 29% less time than high for 1.4 fewer bugs; xhigh found more bugs at much higher cost and time. Medium supports a balanced starting point, not equivalence",
|
|
40
|
+
"status": "current"
|
|
41
|
+
},
|
|
42
|
+
{
|
|
43
|
+
"id": "sonnet-5-5-bug-hunt",
|
|
44
|
+
"profiles": ["balanced"],
|
|
45
|
+
"task": "Find planted bugs: 105 across two repositories, count fixed",
|
|
46
|
+
"model": "claude-sonnet-5-5",
|
|
47
|
+
"harness": "Claude Code",
|
|
48
|
+
"route": "Anthropic first-party API",
|
|
49
|
+
"efforts": [
|
|
50
|
+
{
|
|
51
|
+
"effort": "high",
|
|
52
|
+
"fixed": 34.3,
|
|
53
|
+
"listCostUsd": 19.85,
|
|
54
|
+
"wallMinutes": 27.9
|
|
55
|
+
},
|
|
56
|
+
{
|
|
57
|
+
"effort": "xhigh",
|
|
58
|
+
"fixed": 36.0,
|
|
59
|
+
"listCostUsd": 42.86,
|
|
60
|
+
"wallMinutes": 72.7
|
|
61
|
+
}
|
|
62
|
+
],
|
|
63
|
+
"sourceUrl": "https://bughunt.productcompass.pm/data/benchmark.json",
|
|
64
|
+
"checked": "2026-10-14",
|
|
65
|
+
"samples": "3 runs per effort, non-superseded rows; each run covers both repositories",
|
|
66
|
+
"metric": "planted bugs fixed, mean of 3, out of 105",
|
|
67
|
+
"timeDefinition": "wall clock summed over both repositories; the board says the machine was idle for these runs, so the times do not compare with the Opus entry's",
|
|
68
|
+
"costDefinition": "list-rate API price estimated from usage counts, summed over both repositories; the board notes its harness priced this model on another model's table, so the dollar figure is less certain than the usage counts",
|
|
69
|
+
"limitations": "Same board limits as the Opus entry; its older Sonnet 5 rows are not comparable to these; the board also lists low, medium and max rungs that this entry leaves out",
|
|
70
|
+
"rationale": "At high and xhigh it found 34.3 and 36.0 planted bugs against Opus's 31.7 and 36.0, at similar cost ($19.85 and $42.86 against $22.25 and $34.98); the times are not comparable. Suggest it only where the user allows that model and effort",
|
|
71
|
+
"status": "current"
|
|
72
|
+
},
|
|
73
|
+
{
|
|
74
|
+
"id": "muse-wonder-full-bug-hunt",
|
|
75
|
+
"profiles": ["balanced"],
|
|
76
|
+
"task": "Find planted bugs: 105 across two repositories, count fixed",
|
|
77
|
+
"model": "muse-wonder-full-1.0",
|
|
78
|
+
"harness": "Muse Code (Meta's agent CLI)",
|
|
79
|
+
"route": "Meta API through a Muse Code subscription plan; no per-use price",
|
|
80
|
+
"efforts": [
|
|
81
|
+
{
|
|
82
|
+
"effort": "xhigh",
|
|
83
|
+
"fixed": 41.5,
|
|
84
|
+
"listCostUsd": "unknown",
|
|
85
|
+
"wallMinutes": 90.5
|
|
86
|
+
},
|
|
87
|
+
{
|
|
88
|
+
"effort": "max",
|
|
89
|
+
"fixed": 42,
|
|
90
|
+
"listCostUsd": "unknown",
|
|
91
|
+
"wallMinutes": 145.3
|
|
92
|
+
}
|
|
93
|
+
],
|
|
94
|
+
"sourceUrl": "https://bughunt.productcompass.pm/data/benchmark.json",
|
|
95
|
+
"checked": "2026-10-14",
|
|
96
|
+
"samples": "2 runs per effort (xhigh 44 and 39, max 44 and 40); each run covers both repositories",
|
|
97
|
+
"metric": "planted bugs fixed, mean of 2, out of 105",
|
|
98
|
+
"timeDefinition": "active wall clock summed over both repositories; the board says its machine was shared with other work, and the second run of each effort was resumed in segments across two Muse Code versions, so only its active time is counted",
|
|
99
|
+
"costDefinition": "unknown: Meta publishes no price for this model, so no dollar figure is recorded",
|
|
100
|
+
"limitations": "Two runs per effort, and the board measures a median 1.7-point run-to-run move on three-run configurations, so the 0.5-point gap between xhigh and max is not a difference; the board lists no high or lower rung; the board's own note says max led on one repository and xhigh on the other; reachable only through a Muse Code subscription plan, so it fits a user who already has one; its usage draws on that plan's rolling windows, which the board does not measure",
|
|
101
|
+
"rationale": "xhigh fixed 41.5 planted bugs against Opus 5.5 xhigh's 36.0 and Sonnet 5.5 xhigh's 36.0, in a time the board says is loosely comparable at best, and max added nothing clear for 55 more minutes. Suggest it as an additional seat from a different model family where the user has a Muse Code plan and allows the effort; start at xhigh, and use max only on request",
|
|
102
|
+
"status": "current"
|
|
103
|
+
}
|
|
104
|
+
]
|
|
105
|
+
}
|
|
@@ -0,0 +1,46 @@
|
|
|
1
|
+
# Panel recommendations
|
|
2
|
+
|
|
3
|
+
Load this only during [panel setup](panel-setup.md) or when the user asks for
|
|
4
|
+
recommendations. [`panel-recommendations.json`](panel-recommendations.json) holds
|
|
5
|
+
the entries; this page says how to read them.
|
|
6
|
+
|
|
7
|
+
The entries are dated suggestions for choosing seats, not measured guarantees,
|
|
8
|
+
capability tiers or required seat counts. A model or effort here can be absent from
|
|
9
|
+
a user's catalog, over their limit or excluded by them; the user's limits win.
|
|
10
|
+
|
|
11
|
+
## Profiles
|
|
12
|
+
|
|
13
|
+
| Profile | Intended use | Starting shape |
|
|
14
|
+
| ---------- | --------------------------------------------- | ----------------------------------------------------------- |
|
|
15
|
+
| `quick` | A bounded question with an objective verifier | Two eligible independent models, limited reply scope |
|
|
16
|
+
| `balanced` | A substantive review with explicit criteria | Two models, with an explicitly permitted residual tie-break |
|
|
17
|
+
| `deep` | A broad or difficult review the user selected | A named larger panel, with receipt checks and adjudication |
|
|
18
|
+
|
|
19
|
+
The `quick` and `deep` profiles have no entry yet; current entries are tagged
|
|
20
|
+
`balanced` only. The only recorded evidence here, the Bug Hunt board,
|
|
21
|
+
measures single model-and-effort runs on a find-every-planted-bug task. It does not
|
|
22
|
+
test a two-model panel with a limited reply scope, or a larger panel with receipt
|
|
23
|
+
checks and adjudication, so tagging an entry `quick` or `deep` would extrapolate.
|
|
24
|
+
For those profiles, suggest only the shape in the table and the user's own limits,
|
|
25
|
+
and say that no recorded evidence backs a seat choice.
|
|
26
|
+
|
|
27
|
+
## Reading an entry
|
|
28
|
+
|
|
29
|
+
Each entry names the task the evidence measures, the exact model and effort, the
|
|
30
|
+
harness and route, the source URL, the date it was checked, the number of runs, the
|
|
31
|
+
quality metric and its definitions of time and cost, the limits of the evidence and
|
|
32
|
+
why it is suggested. A value that was not recorded is the string `unknown`.
|
|
33
|
+
|
|
34
|
+
- Compare entries only on the same board and conditions. A board's task time is not
|
|
35
|
+
a host's time to finish.
|
|
36
|
+
- Public list prices are not a user's subscription billing.
|
|
37
|
+
- A `status` other than `current` (`superseded`, `withdrawn`, `unverified`) stays
|
|
38
|
+
listed with its evidence and is not suggested.
|
|
39
|
+
|
|
40
|
+
## Updating
|
|
41
|
+
|
|
42
|
+
Check an entry again when a model, route, price or benchmark correction changes the
|
|
43
|
+
choice, and when preparing a release. Only changed entries need new evidence. An
|
|
44
|
+
update shows the old and suggested choices, the evidence date, the expected
|
|
45
|
+
tradeoff and the uncertainty; the user confirms before any saved preference
|
|
46
|
+
changes, and installing a newer catalog leaves saved profiles and bindings alone.
|