claude-code-superkit 0.0.0-stage → 1.5.3

package/packages/stack-agents/go/references/benchmark-methodology.md

@@ -0,0 +1,280 @@
+# Benchmark Methodology
+
+> Reference document for go-reviewer, go-performance-reviewer. Loaded on demand via Read tool.
+> Upstream: https://pkg.go.dev/testing#hdr-Benchmarks · https://pkg.go.dev/golang.org/x/perf/cmd/benchstat
+
+## Why Methodology Matters
+
+A single benchmark run is a measurement, not an answer. CPUs, filesystems, and the Go scheduler introduce noise that dwarfs changes under ~10%. Without a repeatable methodology, "my change is 30% faster" often means "my last run happened to be fast."
+
+Rules of the game:
+
+1. Run every benchmark with `-count=6` (or more) — one run is noise.
+2. Compare two versions with `benchstat` — it reports p-values, not just percentage deltas.
+3. Quiet machine: close the browser, turn off other builds, plug in the laptop.
+4. Use `-benchmem` always. Allocations are usually the thing that changed.
+
+## `testing.B` Basics
+
+```go
+func BenchmarkUserLookup(b *testing.B) {
+    repo := newRepo(b)
+
+    b.ReportAllocs()   // always; force B/op + allocs/op output
+    b.ResetTimer()     // drop setup cost from the measurement
+
+    for i := 0; i < b.N; i++ {
+        _, err := repo.FindByID(context.Background(), 42)
+        if err != nil { b.Fatal(err) }
+    }
+}
+```
+
+- `b.N` — the framework adjusts this until the benchmark runs for `-benchtime` (default 1s).
+- Write the body so that calling it `b.N` times is meaningful — don't do setup inside the loop.
+- `b.Fatal` / `b.Fatalf` marks the whole benchmark as failed; don't swallow errors.
+
+### Timing Control
+
+```go
+func BenchmarkWithSetup(b *testing.B) {
+    for i := 0; i < b.N; i++ {
+        data := buildHeavyInput(i) // setup cost we don't want to measure
+        b.StartTimer()             // rare — default is running
+        _ = process(data)
+        b.StopTimer()              // stop measuring, keep the loop going
+    }
+}
+```
+
+Prefer `b.ResetTimer` right before the loop and avoid per-iteration StopTimer/StartTimer — the per-call overhead skews results.
+
+### Throughput Reporting
+
+`b.SetBytes(n)` makes the framework report `MB/s`:
+
+```go
+func BenchmarkJSON(b *testing.B) {
+    payload := make([]byte, 4096)
+    b.SetBytes(int64(len(payload)))
+    b.ResetTimer()
+    for i := 0; i < b.N; i++ {
+        _ = json.Unmarshal(payload, &out)
+    }
+}
+// Output: ... 250 MB/s
+```
+
+Use `SetBytes` for I/O, parsing, compression — anything where "X MB/s" is the natural unit.
+
+## Sub-Benchmarks
+
+`b.Run` for parameter sweeps — input sizes, algorithm variants:
+
+```go
+func BenchmarkSort(b *testing.B) {
+    for _, n := range []int{100, 1_000, 10_000, 100_000} {
+        b.Run(fmt.Sprintf("n=%d", n), func(b *testing.B) {
+            data := randInts(n)
+            b.ResetTimer()
+            for i := 0; i < b.N; i++ {
+                cp := slices.Clone(data)
+                slices.Sort(cp)
+            }
+        })
+    }
+}
+```
+
+Output:
+
+```
+BenchmarkSort/n=100-8       500000   2643 ns/op
+BenchmarkSort/n=1000-8       30000  38120 ns/op
+BenchmarkSort/n=10000-8       2500 472900 ns/op
+BenchmarkSort/n=100000-8       200 5.82e+06 ns/op
+```
+
+Use the `/` pattern in `-bench`: `go test -bench=BenchmarkSort/n=1000`.
+
+## Running Benchmarks
+
+```bash
+# Basic — run every Benchmark*, capture memory, 6 samples
+go test -bench=. -benchmem -count=6 ./...
+
+# Focus a single benchmark, longer runtime for smaller-variance numbers
+go test -bench=BenchmarkJSON -benchtime=3s -count=10
+
+# Fixed iteration count (debugging flaky benchmarks)
+go test -bench=. -benchtime=1000x
+
+# CPU sweep — measure at 1/2/4/8 cores
+go test -bench=. -cpu=1,2,4,8
+
+# Alternate package
+go test -bench=. ./pkg/cache/...
+```
+
+Key flags:
+
+| Flag | Purpose |
+|------|---------|
+| `-bench regex` | Which benchmarks to run (`.` = all) |
+| `-benchmem` | Report `B/op` + `allocs/op` — always on |
+| `-count N` | Run each benchmark N times (≥ 6 for stats) |
+| `-benchtime Xs` or `-benchtime Nx` | Target duration OR fixed iterations |
+| `-cpu 1,2,4` | Set `GOMAXPROCS` for each run — finds contention |
+| `-cpuprofile cpu.prof` | Capture CPU profile |
+| `-memprofile mem.prof` | Capture heap profile |
+| `-trace trace.out` | Execution tracer output |
+| `-run ^$` | Skip tests (tests run by default alongside benchmarks) |
+
+## benchstat Workflow
+
+Install: `go install golang.org/x/perf/cmd/benchstat@latest`.
+
+```bash
+# Baseline — your current branch
+go test -run ^$ -bench=BenchmarkProcess -benchmem -count=10 > old.txt
+
+# Apply your change, then:
+go test -run ^$ -bench=BenchmarkProcess -benchmem -count=10 > new.txt
+
+benchstat old.txt new.txt
+```
+
+**Capture the two passes serially on the same quiet machine.** Run the `old.txt` pass, then the `new.txt` pass — never concurrently, and never on different hardware. Concurrent captures reintroduce the CPU/cache/scheduler contention that benchstat's `p`-values exist to remove, silently corrupting the comparison. This is orthogonal to the `-cpu` sweep, which varies `GOMAXPROCS` *within* a single pass.
+
+Output:
+
+```
+                  │    old.txt    │              new.txt               │
+                  │    sec/op     │    sec/op     vs base              │
+Process-8            12.30m ± 2%     8.70m ± 1%  -29.27% (p=0.002 n=10)
+
+                  │    old.txt    │             new.txt              │
+                  │     B/op      │    B/op     vs base              │
+Process-8             4.80Ki ± 0%   1.20Ki ± 0%  -75.00% (p=0.002 n=10)
+```
+
+Read:
+
+- `± N%` — coefficient of variation; under 5% is stable.
+- `vs base` — relative delta; negative is faster / less memory.
+- `p=0.002` — Mann-Whitney U test p-value; `< 0.05` means statistically significant.
+- `n=10` — sample size used (might be trimmed if outliers were rejected).
+
+If `p > 0.05`, benchstat prints `~` — the delta is indistinguishable from noise. Don't claim wins on insignificant results.
+
+## Profiling Alongside Benchmarks
+
+```bash
+go test -run ^$ -bench=BenchmarkProcess -cpuprofile cpu.prof -memprofile mem.prof -count=1
+
+go tool pprof cpu.prof
+# (pprof) top 20
+# (pprof) list ProcessOrder
+# (pprof) web      # flame graph in browser
+```
+
+For long-running benchmarks, also capture a trace:
+
+```bash
+go test -bench=BenchmarkProcess -trace trace.out -benchtime=5s
+go tool trace trace.out
+```
+
+The trace UI shows goroutine scheduling, GC pauses, and syscall blocking — invaluable for diagnosing variance.
+
+## The Dead Code Elimination Trap
+
+Benchmarks that don't use their result get eliminated by the compiler. Classic failure mode: you measure 0.3 ns/op for a function that obviously can't be that fast.
+
+```go
+// WRONG — result unused, loop may be elided
+func BenchmarkHash(b *testing.B) {
+    for i := 0; i < b.N; i++ {
+        hash("hello")
+    }
+}
+
+// RIGHT — assign to a package-level sink
+var sink uint64
+
+func BenchmarkHash(b *testing.B) {
+    var s uint64
+    for i := 0; i < b.N; i++ {
+        s = hash("hello")
+    }
+    sink = s // prevent elimination
+}
+```
+
+Alternatives: pass the result to `b.Log` conditionally (`if sink == 0 { b.Log(sink) }`), or call `runtime.KeepAlive(result)`.
+
+Symptom: nanoseconds per op that don't match the operation cost. `go test -gcflags="-m"` will show inlining / dead-store eliminations.
+
+## Reading Benchmark Output
+
+```
+BenchmarkFoo-8    1000000    1234 ns/op    456 B/op    7 allocs/op
+```
+
+| Column | Meaning |
+|--------|---------|
+| `BenchmarkFoo-8` | Name + `GOMAXPROCS` |
+| `1000000` | Iterations the framework chose (`b.N`) |
+| `1234 ns/op` | Wall-clock nanoseconds per iteration |
+| `456 B/op` | Bytes allocated per iteration (from `-benchmem`) |
+| `7 allocs/op` | Number of heap allocations per iteration |
+
+Derived:
+
+- Ops/sec = `1e9 / ns/op` — handy for throughput claims
+- Allocation pressure = `allocs/op` × call rate — often the real bottleneck on a GC'd language
+- Target "0 allocs/op" for hot paths — it's achievable; profile to find the leak
+
+## Common Pitfalls
+
+| Pitfall | Symptom | Fix |
+|---------|---------|-----|
+| Running with `-count=1` | Can't distinguish signal from noise | `-count=6` minimum |
+| No `benchstat` / just eyeball percentages | False wins on noise | Use `benchstat`; respect `p > 0.05` |
+| Setup inside the loop body | Measures builder+work, not work | Pull setup out; use `b.ResetTimer` |
+| Laptop on battery / thermal throttle | CPU downclocks mid-run | Plug in; close browser; disable Turbo Boost for stability if benchmarking often |
+| `-benchtime=100x` in CI | Each op might take milliseconds; 100 samples is nothing | `-benchtime=2s` or more |
+| Comparing across Go versions or hardware | Apples to oranges | Benchmark on the same machine, same `go version` |
+| Capturing variants concurrently | benchstat still shows a delta, but it is contention, not code | Run the two passes serially |
+| Ignoring `allocs/op` delta | CPU looks equal, but allocations doubled | Always read all three columns |
+| `b.ReportAllocs` missing | `B/op` / `allocs/op` missing from output | Add `b.ReportAllocs()` at the top |
+
+## Methodology Checklist
+
+Before claiming a perf improvement:
+
+- [ ] Ran with `-count=6` or higher
+- [ ] Ran on a quiet machine (closed other heavy apps)
+- [ ] Compared with `benchstat`, saw `p < 0.05`
+- [ ] Inspected `B/op` and `allocs/op` — not just `ns/op`
+- [ ] Added a sink or `runtime.KeepAlive` if the result isn't consumed
+- [ ] Reproduced on a second machine / CI runner
+
+## Review Checklist
+
+When reviewing benchmark code, flag:
+
+- **CRITICAL** — Benchmark result never consumed (`sink` missing) → measuring nothing
+- **WARNING** — `-count=1` assumed in CI scripts or PR templates → no statistical rigor
+- **WARNING** — Setup inside the loop body, no `b.ResetTimer`
+- **WARNING** — Benchmark without `-benchmem` / no `b.ReportAllocs()` → missing allocation story
+- **SUGGESTION** — No sub-benchmarks (`b.Run`) for a parameter sweep that would surface nonlinear cost
+- **SUGGESTION** — Missing `b.SetBytes` in an I/O / parsing benchmark (throughput easier to reason about)
+
+## Further Reading
+
+- `testing` package: https://pkg.go.dev/testing#hdr-Benchmarks
+- `benchstat`: https://pkg.go.dev/golang.org/x/perf/cmd/benchstat
+- Profiling workflow: `packages/stack-agents/go/references/performance-profiling.md`
+- Dave Cheney's "High Performance Go Workshop": https://dave.cheney.net/high-performance-go-workshop/gophercon-2019.html
+- Escape analysis flag: `go build -gcflags="-m"`

package/packages/stack-agents/go/references/channel-patterns.md

@@ -0,0 +1,384 @@
+# Channel Patterns
+
+> Reference document for go-concurrency-reviewer. Loaded on demand via Read tool.
+
+## Fan-Out / Fan-In
+
+Distribute work across multiple goroutines (fan-out), collect results into one channel (fan-in).
+
+```go
+// Fan-out: spawn N workers reading from one input channel
+func fanOut(ctx context.Context, input <-chan Job, workers int) []<-chan Result {
+    outputs := make([]<-chan Result, workers)
+    for i := range workers {
+        out := make(chan Result)
+        outputs[i] = out
+        go func() {
+            defer close(out)
+            for job := range input {
+                select {
+                case <-ctx.Done():
+                    return
+                case out <- process(job):
+                }
+            }
+        }()
+    }
+    return outputs
+}
+
+// Fan-in: merge multiple channels into one
+func fanIn(ctx context.Context, channels ...<-chan Result) <-chan Result {
+    merged := make(chan Result)
+    var wg sync.WaitGroup
+    wg.Add(len(channels))
+
+    for _, ch := range channels {
+        go func() {
+            defer wg.Done()
+            for result := range ch {
+                select {
+                case <-ctx.Done():
+                    return
+                case merged <- result:
+                }
+            }
+        }()
+    }
+
+    go func() {
+        wg.Wait()
+        close(merged)
+    }()
+
+    return merged
+}
+
+// Usage
+input := make(chan Job)
+outputs := fanOut(ctx, input, 4)
+results := fanIn(ctx, outputs...)
+```
+
+## Pipeline
+
+Chain of stages where each stage reads from input, processes, writes to output.
+
+```go
+// Stage 1: generate
+func generate(ctx context.Context, nums ...int) <-chan int {
+    out := make(chan int)
+    go func() {
+        defer close(out)
+        for _, n := range nums {
+            select {
+            case <-ctx.Done():
+                return
+            case out <- n:
+            }
+        }
+    }()
+    return out
+}
+
+// Stage 2: square
+func square(ctx context.Context, in <-chan int) <-chan int {
+    out := make(chan int)
+    go func() {
+        defer close(out)
+        for n := range in {
+            select {
+            case <-ctx.Done():
+                return
+            case out <- n * n:
+            }
+        }
+    }()
+    return out
+}
+
+// Stage 3: filter
+func filter(ctx context.Context, in <-chan int, predicate func(int) bool) <-chan int {
+    out := make(chan int)
+    go func() {
+        defer close(out)
+        for n := range in {
+            if predicate(n) {
+                select {
+                case <-ctx.Done():
+                    return
+                case out <- n:
+                }
+            }
+        }
+    }()
+    return out
+}
+
+// Compose pipeline
+ctx, cancel := context.WithCancel(context.Background())
+defer cancel()
+
+nums := generate(ctx, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10)
+squared := square(ctx, nums)
+even := filter(ctx, squared, func(n int) bool { return n%2 == 0 })
+
+for result := range even {
+    fmt.Println(result) // 4, 16, 36, 64, 100
+}
+```
+
+## Signaling Patterns
+
+### Done Channel
+
+```go
+// Signal completion without data
+done := make(chan struct{})
+
+go func() {
+    defer close(done) // signal by closing
+    doWork()
+}()
+
+<-done // wait for completion
+```
+
+### Context Done
+
+```go
+go func(ctx context.Context) {
+    for {
+        select {
+        case <-ctx.Done():
+            fmt.Println("cancelled:", ctx.Err())
+            return
+        case data := <-dataCh:
+            process(data)
+        }
+    }
+}(ctx)
+```
+
+## Channel Ownership Rules
+
+**Only the sender closes a channel. Never the receiver.**
+
+```go
+// CORRECT — producer owns and closes
+func produce(ctx context.Context) <-chan Item {
+    ch := make(chan Item)
+    go func() {
+        defer close(ch) // producer closes
+        for item := range items {
+            select {
+            case <-ctx.Done():
+                return
+            case ch <- item:
+            }
+        }
+    }()
+    return ch // return read-only channel
+}
+
+// Consumer just ranges — never closes
+for item := range produce(ctx) {
+    process(item)
+}
+```
+
+### Directed Channel Types
+
+Use directed types in function signatures to enforce ownership.
+
+```go
+// Send-only: can write, cannot read
+func producer(out chan<- int) {
+    out <- 42
+    close(out)
+}
+
+// Receive-only: can read, cannot write or close
+func consumer(in <-chan int) {
+    for v := range in {
+        fmt.Println(v)
+    }
+}
+```
+
+## Buffered vs Unbuffered
+
+| Type | Behavior | Use When |
+|------|----------|----------|
+| Unbuffered `make(chan T)` | Send blocks until receiver ready | Synchronization between goroutines |
+| Buffered `make(chan T, n)` | Send blocks only when buffer full | Throughput, decoupling speed differences |
+
+```go
+// Unbuffered — synchronization point
+// Both goroutines must be ready simultaneously
+sync := make(chan struct{})
+go func() {
+    prepareData()
+    sync <- struct{}{} // blocks until main reads
+}()
+<-sync // blocks until goroutine sends
+useData()
+
+// Buffered — throughput
+// Producer can run ahead of consumer
+jobs := make(chan Job, 100)
+go func() {
+    for _, j := range allJobs {
+        jobs <- j // won't block until buffer full
+    }
+    close(jobs)
+}()
+```
+
+### Buffer Size Guidelines
+
+| Size | Use Case |
+|------|----------|
+| 0 (unbuffered) | Synchronization, handoff |
+| 1 | Signal that can be "pending" (e.g., notify channel) |
+| N (known batch) | Pre-buffering known workload |
+| N (worker count) | One slot per worker |
+
+```go
+// Size 1 — "at most one pending notification"
+notify := make(chan struct{}, 1)
+
+// Non-blocking send: notify if not already pending
+select {
+case notify <- struct{}{}:
+default: // already pending, skip
+}
+```
+
+## Select with Default (Non-Blocking)
+
+```go
+// Non-blocking receive
+select {
+case msg := <-ch:
+    process(msg)
+default:
+    // ch not ready, do something else
+}
+
+// Non-blocking send
+select {
+case ch <- msg:
+    // sent
+default:
+    // channel full or not ready, drop or queue
+}
+
+// Try-lock pattern
+select {
+case sem <- struct{}{}: // acquire
+    defer func() { <-sem }() // release
+    doCriticalWork()
+default:
+    // someone else holds it
+    return ErrBusy
+}
+```
+
+## Nil Channel Tricks
+
+A nil channel blocks forever on both send and receive. This is useful in select to dynamically disable cases.
+
+```go
+func merge(ctx context.Context, a, b <-chan int) <-chan int {
+    out := make(chan int)
+    go func() {
+        defer close(out)
+        for a != nil || b != nil {
+            select {
+            case <-ctx.Done():
+                return
+            case v, ok := <-a:
+                if !ok {
+                    a = nil // disable this case
+                    continue
+                }
+                out <- v
+            case v, ok := <-b:
+                if !ok {
+                    b = nil // disable this case
+                    continue
+                }
+                out <- v
+            }
+        }
+    }()
+    return out
+}
+```
+
+### Dynamic Channel Selection
+
+```go
+func processWithTimeout(ch <-chan Data, timeout time.Duration) (Data, error) {
+    var timer <-chan time.Time // nil — won't fire
+    if timeout > 0 {
+        t := time.NewTimer(timeout)
+        defer t.Stop()
+        timer = t.C // now it will fire
+    }
+
+    select {
+    case data := <-ch:
+        return data, nil
+    case <-timer: // skipped entirely if timer is nil
+        return Data{}, ErrTimeout
+    }
+}
+```
+
+## Common Mistakes
+
+### Forgetting to Close
+
+```go
+// BUG — consumer ranges forever
+func produce() <-chan int {
+    ch := make(chan int)
+    go func() {
+        for i := range 10 {
+            ch <- i
+        }
+        // forgot close(ch) — consumer blocks forever after 10 items
+    }()
+    return ch
+}
+```
+
+### Closing from Receiver Side
+
+```go
+// BUG — panic if sender writes after close
+go func() {
+    for data := range input {
+        process(data)
+    }
+    close(input) // WRONG: receiver should not close
+}()
+```
+
+### Range Over Non-Closed Channel
+
+```go
+// BUG — deadlock if channel is never closed
+for v := range ch { // blocks forever after last value
+    process(v)
+}
+```
+
+## When to Use
+
+Apply when reviewing channel usage and concurrent communication patterns. Flag violations as:
+- **CRITICAL**: Receiver closing channel, unbuffered channel in goroutine without exit path, range over channel that's never closed
+- **WARNING**: Missing context/done in select, oversized buffer without justification, send on closed channel risk
+- **SUGGESTION**: Could use directed channel types, nil channel trick for dynamic disable, pipeline pattern for data processing