heapscope 0.6.0

data/docs/diagnostics/HS001_persistent_class_growth.md

@@ -0,0 +1,39 @@
+# HS001 — persistent_class_growth
+
+## What it means
+
+A class's live object count increased across the measured window or retention cycles.
+
+## Evidence used
+
+- Class counts before/after (or multi-sample series)
+- Growth pattern (`monotonic_growth`, `linear_growth`, `exponential_like`)
+- Optional forced-GC between samples
+
+## Likely causes
+
+- Unbounded registry / array / hash
+- Request or job objects retained globally or in thread-locals
+- Memoization without eviction
+- Callbacks retaining contexts
+
+## False positives
+
+- Cache warmup / lazy loading
+- First-time autoload in development
+- Batch size increases that are intentional
+- Delayed GC (mitigate with `force_gc: true` in experiments)
+
+## How to investigate
+
+1. Run `HeapScope.retention_test` with forced GC
+2. Enable allocation tracing for the class's sites
+3. Inspect thread-locals, globals, and constants
+4. Use deep mode retention paths for sampled objects
+
+## Potential fixes
+
+- Clear request/job context in `ensure`
+- Bound caches / use TTL or LRU
+- Unsubscribe listeners
+- Avoid capturing large objects in long-lived closures

data/docs/diagnostics/HS002_high_retention_ratio.md

@@ -0,0 +1,24 @@
+# HS002 — high_retention_ratio
+
+## What it means
+
+A large fraction of objects allocated during the workload remained reachable afterward.
+
+## Evidence used
+
+- `total_allocated_objects` / `total_freed_objects` deltas
+- Surviving estimate and retention ratio
+
+## Likely causes
+
+- Sticky object graphs retained by roots
+- Accidental global retention of request data
+
+## False positives
+
+- Workload that intentionally builds long-lived structures
+- Insufficient GC between measurements
+
+## Investigation / fixes
+
+Use class deltas + retention sessions; clear accidental roots; bound collections.

data/docs/diagnostics/HS003_thread_local_retention.md

@@ -0,0 +1,24 @@
+# HS003 — thread_local_retention
+
+## What it means
+
+Retention appears consistent with thread-local (or fiber-local) storage holding growing state — common under Puma/Sidekiq thread reuse.
+
+## Evidence used
+
+- Thread key inventory
+- Growing populations reachable from thread contexts
+- Persistence across request/job cycles
+
+## Likely causes
+
+- `Thread.current[:context]` not cleared
+- Middleware storing request state on the worker thread
+
+## False positives
+
+- Intentional per-thread caches with bounds
+
+## Fixes
+
+Clear thread/fiber locals in `ensure` after each request/job; avoid storing full request graphs.

data/docs/diagnostics/HS004_unbounded_collection.md

@@ -0,0 +1,19 @@
+# HS004 — unbounded_collection
+
+## What it means
+
+An Array/Hash/Set-like structure grows across samples without observed shrinkage after GC.
+
+## Evidence used
+
+- Size series across samples
+- Owner class when identifiable
+
+## False positives
+
+- Bounded caches still filling toward max
+- Warmup periods
+
+## Fixes
+
+Cap size, eviction, periodic compaction, or stop appending.

data/docs/diagnostics/HS005_callback_accumulation.md

@@ -0,0 +1,14 @@
+# HS005 — callback_accumulation
+
+## What it means
+
+Listeners, subscribers, or observer lists grow over time (often repeated registration without unsubscribe).
+
+## Evidence used
+
+- Listener count series (via adapters)
+- Proc/Array growth near event buses
+
+## Fixes
+
+Unsubscribe on teardown; register once at boot; use weak references where appropriate.

data/docs/diagnostics/HS006_closure_retention.md

@@ -0,0 +1,15 @@
+# HS006 — closure_retention
+
+## What it means
+
+Long-lived `Proc` objects appear to retain large object graphs via captured bindings.
+
+## Evidence used
+
+- Proc population growth
+- Approximate retained size from bounded traversal
+- Allocation site of the Proc when tracing is enabled
+
+## Caveats
+
+Runtime APIs may not expose closure captures cleanly — treat as hypothesis.

data/docs/diagnostics/HS007_poor_gc_recovery.md

@@ -0,0 +1,18 @@
+# HS007 — poor_gc_recovery
+
+## What it means
+
+After opt-in forced GC, live populations or RSS remain elevated versus baseline.
+
+## Evidence used
+
+- Before / after GC / after idle phase snapshots
+
+## False positives
+
+- Native memory / fragmentation (see HS010)
+- Intentional caches filled during workload
+
+## Note
+
+Forced GC distorts production behavior — experiments only.

data/docs/diagnostics/HS008_baseline_regression.md

@@ -0,0 +1,15 @@
+# HS008 — baseline_regression
+
+## What it means
+
+Current retention exceeds a stored baseline beyond a configured threshold.
+
+## Evidence used
+
+- Baseline retained objects/bytes vs current report
+
+## Use in CI
+
+```bash
+heapscope compare baseline.json current.json --threshold 0.5
+```

data/docs/diagnostics/HS009_high_allocation_pressure.md

@@ -0,0 +1,13 @@
+# HS009 — high_allocation_pressure
+
+## What it means
+
+Many objects were allocated and mostly freed — **churn**, not necessarily a leak.
+
+## Evidence used
+
+- High allocated Δ with high freed Δ / low retention ratio
+
+## Action
+
+Optimize hot allocation sites if GC/CPU time matters; do not treat as retention failure.

data/docs/diagnostics/HS010_native_memory_mismatch.md

@@ -0,0 +1,19 @@
+# HS010 — native_memory_mismatch
+
+## What it means
+
+Process RSS grew substantially while the Ruby heap (live slots / object estimates) did not.
+
+## Evidence used
+
+- RSS Δ vs heap live slots Δ and shallow byte estimates
+
+## Likely causes
+
+- Native extension allocations
+- Allocator fragmentation (glibc / jemalloc / macOS)
+- mmap / external buffers
+
+## Important
+
+**RSS growth ≠ Ruby object leak.** Do not blame object retention from this signal alone.

data/docs/guides/ci-budgets.md

@@ -0,0 +1,43 @@
+# Guide: CI memory budgets
+
+## Presets (0.6+)
+
+```ruby
+HeapScope.budget_preset(:rails_request)  # typical request cycle
+HeapScope.budget_preset(:sidekiq_job)    # background job
+HeapScope.budget_preset(:ci_strict)      # tight PR gate
+```
+
+```ruby
+report = HeapScope.check_budget(budget: HeapScope.budget_preset(:ci_strict)) { scenario }
+raise report.budget_result[:violations].join("\n") unless report.passed_budget?
+```
+
+## Capture a baseline on `main`
+
+```bash
+bundle exec ruby script/memory_scenario.rb # writes report.json
+heapscope baseline create report.json -o baseline.json
+git add baseline.json
+```
+
+## On PRs
+
+```bash
+heapscope compare baseline.json report.json --threshold 0.5
+```
+
+Exit code `1` means regression.
+
+## Hand-tuned budgets
+
+```ruby
+budget = HeapScope::Budget.new(
+  max_retained_objects: 2_000,
+  max_rss_growth: 20 * 1024 * 1024,
+  severity_threshold: :high
+)
+report = HeapScope.check_budget(budget: budget) { scenario }
+```
+
+Keep thresholds noisy-tolerant; prefer `experiment(runs: 5)` medians for flaky heaps.

data/docs/guides/first-retention-experiment.md

@@ -0,0 +1,25 @@
+# Guide: First retention experiment
+
+Goal: prove whether a workload **retains** objects across GC, not merely allocates.
+
+## Steps
+
+1. Warm up the app (routes, autoload, DB connections).
+2. Capture a baseline mental model: what *should* survive?
+3. Run:
+
+```ruby
+report = HeapScope.retention_test(cycles: 10, force_gc: true) do
+  50.times { MyEndpoint.call }
+end
+puts report
+report.save_html("retention.html")
+```
+
+4. Read findings in order: facts → derived → hypothesis → suspected cause.
+5. If HS001/HS002 fire, enable allocation tracing and re-run once.
+6. Inspect thread-locals if HS003 appears.
+
+## Interpreting HEALTHY
+
+Temporary growth that reclaims after forced GC is success. Do not chase HS009 (churn) as a leak.

data/docs/guides/production-safe.md

@@ -0,0 +1,17 @@
+# Guide: Production-safe sampling
+
+```ruby
+HeapScope.configure do |c|
+  c.mode = :production_safe
+  c.object_sample_rate = 0.05
+end
+
+use HeapScope::Middleware, sample_rate: 0.01, mode: :lightweight
+```
+
+Rules of thumb:
+
+- Never force GC in production paths
+- Prefer `lightweight` snapshots
+- Ship reports to local disk / log drains you already trust — HeapScope itself uploads nothing
+- Treat HTML/JSON outputs as sensitive

data/docs/index.md

@@ -0,0 +1,38 @@
+<p align="center">
+  <img src="../assets/wordmark.svg" alt="HeapScope" width="420"/>
+</p>
+
+# Documentation hub
+
+**Website:** [theworker02.github.io/heapscope](https://theworker02.github.io/heapscope/) · **Sponsor:** [thanks.dev/u/gh/theworker02](https://thanks.dev/u/gh/theworker02)
+
+## Start here
+
+| Doc | Purpose |
+|-----|---------|
+| [README](../README.md) | Product overview & quick start |
+| [First retention experiment](guides/first-retention-experiment.md) | Hands-on workflow |
+| [CI budgets](guides/ci-budgets.md) | Regression gates + presets |
+| [Production-safe sampling](guides/production-safe.md) | Low-overhead prod use |
+| [CLI reference](cli.md) | Command catalog |
+| [API overview](api/overview.md) | Public Ruby API |
+| [Diagnostic codes](diagnostics/) | HS001–HS010 encyclopedia |
+| [Allocators](allocators.md) | RSS vs Ruby heap caveats |
+| [Roadmap](ROADMAP.md) | What's next |
+| [ADRs](adr/) | Architecture decisions |
+| [Changelogs](changelogs/) | Release narratives |
+
+## Design pledges
+
+1. **Retention over allocation volume**
+2. **Evidence over certainty**
+3. **Local-only privacy by default**
+4. **Safe degradation across Ruby engines**
+
+## Version
+
+Docs track **HeapScope 0.6.x**.
+
+<p align="center">
+  <img src="../assets/logo.svg" alt="" width="48"/>
+</p>

data/examples/cache_vs_leak.rb

@@ -0,0 +1,30 @@
+# frozen_string_literal: true
+
+require "bundler/setup"
+require "heapscope"
+
+# Bounded cache vs unbounded registry — HeapScope should treat them differently.
+CACHE = {}
+CACHE_MAX = 100
+REGISTRY = []
+
+def fill_cache
+  300.times { |i| CACHE[i % CACHE_MAX] = "v#{i}" }
+end
+
+def fill_registry
+  150.times { REGISTRY << ("row-" + rand(10_000).to_s) }
+end
+
+puts "=== CACHE (expect plateau / low suspicion) ==="
+cache_report = HeapScope.retention_test(cycles: 5, force_gc: true, mode: :lightweight) { fill_cache }
+puts cache_report.summary.inspect
+puts cache_report.findings.map { |f| "#{f.code}/#{f.severity}" }.inspect
+
+puts
+puts "=== REGISTRY (expect persistent growth) ==="
+reg_report = HeapScope.retention_test(cycles: 5, force_gc: true, mode: :lightweight) { fill_registry }
+puts reg_report.summary.inspect
+puts reg_report.findings.map { |f| "#{f.code}/#{f.severity} #{f.subject}" }.inspect
+
+reg_report.save_html("examples/cache_vs_leak.html")

data/examples/closure_capture.rb

@@ -0,0 +1,20 @@
+# frozen_string_literal: true
+
+require "bundler/setup"
+require "heapscope"
+
+# Closure capture demo — enable closure detection explicitly.
+HOLDERS = []
+big = Array.new(2_000) { "payload" }
+
+HeapScope.configure do |c|
+  c.detect_closures = true
+end
+
+report = HeapScope.retention_test(cycles: 4, force_gc: true, mode: :lightweight) do
+  30.times { HOLDERS << -> { big.size } }
+end
+
+puts report
+puts "Closures sampled: #{Array(report.closures).size}"
+report.save_markdown("examples/closure_report.md")

data/examples/healthy_churn.rb

@@ -0,0 +1,16 @@
+# frozen_string_literal: true
+
+require "bundler/setup"
+require "heapscope"
+
+# Healthy temporary allocation — should reclaim after GC.
+report = HeapScope.measure(force_gc: true, mode: :lightweight) do
+  data = []
+  10_000.times { |i| data << { i: i, body: "x" * 20 } }
+  data.clear
+  nil
+end
+
+# Framework/Hash noise from the profiler itself is ignored in findings by default.
+puts report
+puts "Healthy?=#{report.summary[:healthy].inspect} findings=#{report.findings.map(&:code)}"

data/examples/heapscope.yml

@@ -0,0 +1,12 @@
+heapscope:
+  mode: standard
+  track_allocations: false
+  max_graph_depth: 6
+  max_objects: 100000
+  object_sample_rate: 1.0
+  ignore_patterns:
+    - "^Zeitwerk"
+    - "^Bootsnap"
+    - "^HeapScope::"
+  retention:
+    min_cycles: 3

data/examples/import_leak.rb

@@ -0,0 +1,19 @@
+# frozen_string_literal: true
+
+require "bundler/setup"
+require "heapscope"
+
+# Demo: intentional leak via a growing array.
+LEAK = []
+
+puts "HeapScope #{HeapScope::VERSION}"
+puts HeapScope.capabilities
+puts
+
+report = HeapScope.retention_test(cycles: 5, force_gc: true, mode: :lightweight) do
+  200.times { LEAK << (+"item") << ("x" * 50) }
+end
+
+puts report
+report.save("examples/import_leak_report.json")
+puts "Wrote examples/import_leak_report.json"

data/examples/probe_and_session.rb

@@ -0,0 +1,22 @@
+# frozen_string_literal: true
+
+require "bundler/setup"
+require "heapscope"
+
+puts "Probe demo"
+result = HeapScope.probe(title: "temporary objects", force_gc: true, mode: :lightweight) do
+  data = []
+  5_000.times { |i| data << { i: i } }
+  data.clear
+  nil
+end
+puts result.scorecard.to_text
+
+puts
+puts "Session demo"
+session = HeapScope.session("demo-probe")
+session.measure(label: "churn", force_gc: true, mode: :lightweight) do
+  1_000.times { "x" * 10 }
+end
+puts "Session artifacts in .heapscope/sessions/demo-probe/"
+puts "Latest: #{session.latest.inspect}"

data/examples/thread_local_leak.rb

@@ -0,0 +1,31 @@
+# frozen_string_literal: true
+
+require "bundler/setup"
+require "heapscope"
+
+# Demonstrates thread-local retention across "requests" on one thread.
+HeapScopeFixtures = Module.new unless defined?(HeapScopeFixtures)
+
+module ThreadLocalDemo
+  def self.handle_request
+    Thread.current[:request_context] ||= { presenters: [] }
+    50.times do
+      Thread.current[:request_context][:presenters] << ("Presenter:" + ("x" * 40))
+    end
+  end
+
+  def self.clear
+    Thread.current[:request_context] = nil
+  end
+end
+
+ThreadLocalDemo.clear
+
+report = HeapScope.retention_test(cycles: 6, force_gc: true, mode: :lightweight) do
+  ThreadLocalDemo.handle_request
+end
+
+puts report
+report.save("examples/thread_local_report.json")
+report.save_html("examples/thread_local_report.html")
+puts "Wrote examples/thread_local_report.{json,html}"

data/exe/heapscope

@@ -0,0 +1,7 @@
+#!/usr/bin/env ruby
+# frozen_string_literal: true
+
+require "heapscope"
+require "heapscope/cli"
+
+exit HeapScope::CLI.run(ARGV)

data/heapscope.gemspec

@@ -0,0 +1,73 @@
+# frozen_string_literal: true
+
+require_relative "lib/heapscope/version"
+
+Gem::Specification.new do |spec|
+  spec.name = "heapscope"
+  spec.version = HeapScope::VERSION
+  spec.authors = ["theworker02"]
+  spec.email = ["matthewlooney5@gmail.com"]
+
+  spec.summary = "Ruby object retention, heap growth, and memory leak diagnostics"
+  spec.description = <<~DESC
+    HeapScope is a local, evidence-driven memory diagnostics toolkit for Ruby.
+    It helps identify object retention, abnormal heap growth, allocation hot spots,
+    long-lived objects, GC-surviving populations, and likely memory leaks in
+    long-running Ruby processes — without claiming a leak unless evidence is strong.
+  DESC
+  spec.homepage = "https://github.com/theworker02/heapscope"
+  spec.license = "MIT"
+  spec.required_ruby_version = ">= 3.1.0"
+
+  spec.metadata["homepage_uri"] = spec.homepage
+  spec.metadata["source_code_uri"] = "#{spec.homepage}.git"
+  spec.metadata["changelog_uri"] = "#{spec.homepage}/blob/main/CHANGELOG.md"
+  spec.metadata["bug_tracker_uri"] = "#{spec.homepage}/issues"
+  spec.metadata["documentation_uri"] = "https://theworker02.github.io/heapscope/"
+  spec.metadata["funding_uri"] = "https://thanks.dev/u/gh/theworker02"
+  spec.metadata["rubygems_mfa_required"] = "true"
+
+  spec.post_install_message = <<~MSG
+
+    HeapScope #{HeapScope::VERSION} installed.
+      heapscope doctor
+      heapscope about
+      https://theworker02.github.io/heapscope/
+      https://thanks.dev/u/gh/theworker02
+
+    Local-only diagnostics. No telemetry. No object-value dumps by default.
+  MSG
+
+  spec.files = Dir.chdir(__dir__) do
+    `git ls-files -z`.split("\x0").reject do |f|
+      f.start_with?(*%w[bin/ test/ spec/ features/ .git .github benchmark/ site/]) ||
+        f.end_with?(".gem")
+    end
+  end
+  # Ensure packaging works before first commit
+  if spec.files.empty?
+    spec.files = Dir[
+      "lib/**/*",
+      "exe/*",
+      "assets/**/*",
+      "docs/**/*",
+      "examples/**/*",
+      "LICENSE*",
+      "README*",
+      "CHANGELOG*",
+      "CONTRIBUTING*",
+      "CODE_OF_CONDUCT*",
+      "SECURITY*",
+      "heapscope.gemspec"
+    ]
+  end
+  spec.files.reject! { |f| f.start_with?("site/") }
+
+  spec.bindir = "exe"
+  spec.executables = ["heapscope"]
+  spec.require_paths = ["lib"]
+
+  spec.add_development_dependency "minitest", "~> 5.20"
+  spec.add_development_dependency "rake", "~> 13.0"
+  spec.add_development_dependency "rubocop", "~> 1.60"
+end