audition 0.3.0 → 0.4.1

checksums.yaml

@@ -1,7 +1,7 @@
 ---
 SHA256:
-  metadata.gz: 28a9914f63873fd6bc4ec2e86154aa69e4e3dbfb08c3d9ee6218fc521e31836b
-  data.tar.gz: e1033a810ea825b21980fb3cb406bfd7bda1bf69e970e82c988fb0d149c0d172
+  metadata.gz: c47dfccb03a9d164583e3ae0a5530c74a41cc176f62865807fa4a4d1ed11d7f4
+  data.tar.gz: 75697dacb090c469d8f2966d5174fb08186e1808b85737872b0db5f6034a688d
 SHA512:
-  metadata.gz: 07b6250748bf1d3e854157b6ac81f8a2574259f77d2e28746d241e880763cb446410b27736740c29f803e4936fed3d5ff0f6ca1b95d73cc6cb664896630c17fe
-  data.tar.gz: ab564046d55d28d6e94af9e045991574322e14074e4121e788c6b59f8ce3628f4499351d5f9cac7e670564bc9ea7472b3615056d9b91beaeb64dc3e131babc04
+  metadata.gz: a631d1f7cd378ff04c47dcffb9608d332bfb048fc37dbda66179b085c487e936206359b3ea61a1e8fb5c70d121b6b7d94275b2e6d436f32772e1ea68a540fbd4
+  data.tar.gz: 974bbcd0a531588edf1d2474e9c8afb7bd5d3869a985b483ebb2c628007837ed20e50d10a580f6f0204db95e42128bb3de061237bbdaeab28f3d7008d7838a37

data/README.md

@@ -2,7 +2,7 @@
 
 Point it at a Ruby script, a gem, a Rack app, or a Rails root and it
 tells you whether that code can run inside Ractors, why it cannot,
-and how to fix it. Unlike a linter, audition does not stop at
+and how to fix it. Unlike a linter, Audition does not stop at
 pattern-matching your source: whole-program analysis is powered by
 [rubydex](https://github.com/Shopify/rubydex), Shopify's Ruby
 indexer, and the target is also loaded in a sandboxed subprocess
@@ -28,12 +28,13 @@ core itself is being ractorized; see the
   (which rule of the Ractor model it violates) and a `fix`
   (what to write instead).
 - **`--fix` like RuboCop, in two tiers.** Safe corrections:
-  `.freeze` on string constants, sentinels, and containers whose
-  elements are all shareable, `Ractor.make_shareable(...)` for
-  the remaining mutable and shallow-frozen containers and for
-  Proc constants, and
-  boot-time hoisting of method-body requires. `--fix-unsafe` adds
-  semantics-affecting rewrites: magic-comment insertion,
+  `.freeze` on string constants (literals as well as the fresh
+  strings core methods return), sentinels, and containers whose
+  elements are all shareable, and boot-time hoisting of
+  method-body requires. `--fix-unsafe` adds semantics-affecting
+  rewrites: `Ractor.make_shareable(...)` for the remaining mutable
+  and shallow-frozen containers, for containers a core method
+  allocates, and for Proc constants, magic-comment insertion,
   freeze-on-memoize for class-level memoization (both `@x ||=`
   and `return @x if defined?(@x)` idioms keep their caching, the
   memoized value becomes shareable, Rails-core style;
@@ -47,7 +48,7 @@ core itself is being ractorized; see the
   state, so `globalid` is not blamed for ActiveSupport's state.
 - **Dogfooding.** The scanner for Ractor compatibility is built
   using Ractors: static analysis fans out across CPU cores on
-  Ractor workers, and audition passes its own audit. It runs on
+  Ractor workers, and Audition passes its own audit. It runs on
   itself on every commit (a lefthook pre-commit over the staged
   files) and on every push (a full self-audit in CI).
 - **Trained on Rails core.** Several checks and fix suggestions
@@ -58,26 +59,24 @@ core itself is being ractorized; see the
   findings are documented in
   [docs/rails_core_best_practices.md](docs/rails_core_best_practices.md).
 - **Terminal-native output.** Colors, glyphs, and OSC 8 hyperlinks;
-  `path:line` is clickable in supporting terminals. JSON output for
-  CI.
+  `path:line` is clickable in supporting terminals. Long scans
+  narrate their phases on stderr. JSON output for CI.
 
 ```console
 $ audition worker.rb
-* audition 0.1.0 ruby 4.0.6 · script at .
+* Audition 0.3.0 ruby 4.0.6 · script at .
 
   worker.rb
-    x raises inside a Ractor: Ractor::IsolationError: can not
-      access global variable $jobs from non-main Ractor
-      why: The script ran fine on the main Ractor but failed under
-      Ractor.new; the static findings usually pinpoint the line.
-    x worker.rb:1  write to global variable $jobs
-      why: Non-main Ractors cannot access global variables; this
-      raises Ractor::IsolationError the moment the line executes
-      in a Ractor (verified on Ruby 4.0).
-      fix: Pass the value into the Ractor explicitly
-      (Ractor.new(value) { |v| ... }) or over a Ractor::Port; for
-      per-Ractor state use Ractor.current[:key].
-    x worker.rb:4  read of global variable $jobs
+    x worker.rb:1  write to global variable $jobs global-variables
+      why: Non-main Ractors cannot access global variables; this raises
+      Ractor::IsolationError the moment the line executes in a Ractor
+      (verified on Ruby 4.0).
+      fix: Pass the value into the Ractor explicitly (Ractor.new(value) {
+      |v| ... }) or over a Ractor::Port; for per-Ractor state use
+      Ractor.current[:key]; do one-time process setup on the main Ractor
+      before spawning.
+    x worker.rb:1  raises inside a Ractor: Ractor::IsolationError ...
+    x worker.rb:4  read of global variable $jobs global-variables
       ...
 
   dynamic probes
@@ -93,16 +92,29 @@ And the whole-bundle view:
 
 ```console
 $ audition Gemfile.lock --static-only
-╭───────────────┬─────────┬───────────┬────────┬──────────┬─────────╮
-│ gem           │ version │ verdict   │ errors │ warnings │ fixable │
-├───────────────┼─────────┼───────────┼────────┼──────────┼─────────┤
-│ activesupport │ 8.1.0   │ not ready │    157 │       97 │      77 │
-│ i18n          │ 1.14.7  │ not ready │     48 │       40 │      45 │
-│ mail          │ 2.9.1   │ not ready │     27 │        4 │      13 │
-│ rack          │ 3.2.6   │ not ready │     23 │       45 │      60 │
-│ ...           │         │           │        │          │         │
-╰───────────────┴─────────┴───────────┴────────┴──────────┴─────────╯
-0 of 11 gems ractor-ready
+╭───────────────────────────────────────────────────────────────────────────────────────╮
+│                                 Audition bundle sweep                                 │
+├──────────┬──────────┬─────────────┬────────┬────────────┬──────────┬─────────┬────────┤
+│ gem      │ version  │ verdict     │ errors │ dep errors │ warnings │ fixable │ status │
+├──────────┼──────────┼─────────────┼────────┼────────────┼──────────┼─────────┼────────┤
+│ rubocop  │ 1.88.2   │ x not ready │    194 │ -          │      203 │      28 │ ok     │
+│ parser   │ 3.3.12.0 │ x not ready │    154 │ -          │       26 │      26 │ ok     │
+│ rack     │ 3.2.7    │ x not ready │     31 │ -          │       61 │      13 │ ok     │
+│ rake     │ 13.4.2   │ x not ready │     30 │ -          │       24 │       5 │ ok     │
+│ prism    │ 1.9.0    │ x not ready │     16 │ -          │       88 │      15 │ ok     │
+│ json     │ 2.21.2   │ x not ready │      7 │ -          │        5 │       - │ ok     │
+│ pastel   │ 0.8.0    │ x not ready │      2 │ -          │        1 │       - │ ok     │
+│ tty-link │ 0.2.0    │ x not ready │      1 │ -          │        1 │       - │ ok     │
+╰──────────┴──────────┴─────────────┴────────┴────────────┴──────────┴─────────┴────────╯
+
+x 0 of 8 gems ractor-ready · 8 not ready
+```
+
+Verdicts are colored by severity, clean counts are left blank, and
+the sweep names each gem as it finishes:
+
+```console
+◆ Audition sweep rubocop 8/8 (100%) 5.8s
 ```
 
 **Requires Ruby 4.0 or newer**, strictly: the tool targets the modern
@@ -172,6 +184,20 @@ Useful flags:
 | `--capabilities` | table of what this Ruby allows in Ractors |
 | `--timeout 60` | dynamic probe budget in seconds |
 | `--plain` | no colors or hyperlinks (also via NO_COLOR, pipes) |
+| `--progress` / `--no-progress` | force / suppress the stderr narration |
+| `-j 4` / `--workers 4` | scan Ractors (default: cores, capped by `RUBY_MAX_CPU`) |
+
+Long runs narrate themselves on stderr, so stdout stays pipeable:
+a rewriting status line on a terminal, one line per phase off it.
+
+```
+◆ Audition checking 50/919 5% (0.2s, on 8 ractors)
+```
+
+A phase running in Ractors says how many. The narration turns on
+by itself for a large tree or a bundle sweep and stays off for
+`--format json` and `--format github`; the flags force either
+way.
 
 Exit codes: `0` clean, `1` findings at or above the `--fail-on`
 threshold (or a failed dynamic probe), `2` usage error.
@@ -210,11 +236,20 @@ timeout: 60
 exclude:
   - legacy/**
   - db/schema.rb
+test_dirs:
+  - qa
 checks:
   disable:
     - at-exit
 ```
 
+`test_dirs` names the directories that hold tests rather than
+code a production boot loads; findings in them are tagged
+`test` and rated as test code. It defaults to `test`, `spec`,
+`features`, and replaces that list rather than adding to it—the
+`_test.rb` and `_spec.rb` suffixes always count, whatever it
+says.
+
 ## CI and git hooks
 
 **GitHub Actions.** `--format github` turns findings into
@@ -260,7 +295,7 @@ With [pre-commit](https://pre-commit.com):
 - repo: local
   hooks:
     - id: audition
-      name: audition
+      name: Audition
       language: system
       entry: audition --static-only --plain
       types: [ruby]
@@ -289,14 +324,29 @@ Static, with file:line precision:
 - **Class variables**, resolved on the rubydex graph.
 - **Class-level instance variables**, unified across the class
   body, `def self.`, and `class << self`, across files; the classic
-  `@cache ||= {}` and `return @x if defined?(@x)` memoizations.
+  `@cache ||= {}` and `return @x if defined?(@x)` memoizations. A
+  memo whose write is proxied to the main Ractor
+  (`@x || on_main(self) { @x ||= ... }`) is reported as that
+  escape hatch, not as a raw memo.
 - **Constants that are not deeply shareable**: bare mutable
   literals, interpolated strings, the subtle shallow freeze
-  (`[[1], [2]].freeze` still raises; audition explains why), and
-  call results the magic comment never covers (`X.tr(":", "")`,
-  `Regexp.new`, `Regexp.union`, `format`), the shapes Rails fixed
-  last in its own ractorization. Honors `# frozen_string_literal:`
-  and `# shareable_constant_value:` magic comments.
+  (`[[1], [2]].freeze` still raises; Audition explains why), and
+  call results the magic comment never covers, from the
+  return-type contracts of core methods: fresh strings
+  (`X.tr(":", "")`, `[8, 2, 0].join(".")`, `Regexp.new`,
+  `format`), fresh containers (`TYPES.keys`, `LIST.map { }`,
+  `BASE + [:x]`, `DEFAULTS.merge(...)`, `.dup`), string splitters
+  under a shallow freeze (`".*".chars.freeze`), and Method
+  objects, which no freeze makes shareable. A spec executes the
+  tables against the running Ruby. Integer arithmetic,
+  comparisons, and negation are recognized as shareable. Honors
+  `# frozen_string_literal:` and `# shareable_constant_value:`
+  magic comments.
+- **Instance memoization on classes that get frozen**: a lazy
+  `@x ||=` on a class whose initialize freezes self raises
+  FrozenError on first use; on a class with a `freeze` override
+  the memo must be warmed inside the override before `super`
+  (compute on freeze), and one left cold is reported.
 - **Sync primitives and Procs in constants** (Mutex, Queue,
   lambdas), including `Hash.new { }` default procs, which stay
   unshareable even after `.freeze`.
@@ -307,7 +357,12 @@ Static, with file:line precision:
 - **Runtime require and autoload** (serializes all Ractors through
   the main-Ractor proxy).
 - **`Ractor.new` blocks capturing outer locals** (the ArgumentError
-  at creation time), resolved through Prism's exact scope depths.
+  at creation time), resolved through Prism's exact scope depths;
+  and **blocks that `Ractor.shareable_proc` would refuse**, handed
+  to it directly or to a Rails callback macro (`before_create`,
+  `validate`, `on_load`, ...), because a captured local holds a
+  provably unshareable value (`prefix = +"Draft: "`) or is
+  assigned more than once.
 - **Hostile or removed APIs**: `Ractor.yield`/`take` (gone in 4.0),
   ActiveSupport `cattr_*`/`mattr_*` class variables (with the
   `class_attribute` migration Rails itself made) and
@@ -317,20 +372,35 @@ Static, with file:line precision:
   scan of every compiled `.bundle`/`.so` for the
   `rb_ext_ractor_safe` import, which also covers precompiled
   platform gems that ship no sources; an unbuilt checkout is
-  scanned at the source level (`ext/**`, C, Rust, or Zig) instead.
+  scanned at the source level instead (C, Rust, or Zig sources
+  beside their `extconf.rb`, `Cargo.toml` or `build.zig`).
   A silent extension raises `Ractor::UnsafeError` on every call
   from a non-main Ractor, so it rates a warning; a declared one
   gets an info note, because the declaration is the maintainer's
   assertion, not a proof.
+- **The static pass's own blind spots**: where rubydex reports an
+  expression it could not resolve and the shape could hide what
+  the checks above look for, the hole is reported rather than
+  read as a clean line — a singleton opened on a runtime receiver
+  that writes class-level state, a superclass or `include`
+  argument computed at runtime, a constant assigned through an
+  unresolved path. A clean report for such a class covers only
+  what the class itself declares; the dynamic probe reaches the
+  rest.
 
 Dynamic, on the live object graph:
 
 - Runs scripts inside a real Ractor (via `load`, which is not
   proxied) and reports the actual exception.
-- Requires a library, then sweeps every constant it introduced with
+- Requires a library, then sweeps every constant it introduced,
+  under new namespaces and pre-existing ones alike, with
   `Ractor.shareable?`, and inspects every class and module for
   class-level ivars and class variables, with
-  `const_source_location` attribution.
+  `const_source_location` attribution. What the sweep proves
+  shareable retires the static guesses about the same objects:
+  an unproven constant warning disappears, and class-level state
+  that held only shareable values after boot keeps an info note
+  instead of an error.
 - Records every compiled extension the load pulled in,
   dependencies included, and byte-scans each for the
   `rb_ext_ractor_safe` import; silent ones are reported against
@@ -340,13 +410,19 @@ Dynamic, on the live object graph:
   the per-worker model of Ractor web servers; then hammers it from
   4 Ractors x 25 requests to surface failures that only appear
   under concurrency.
-- Boots Rails (`config/environment.rb`), eager-loads, and sweeps
-  the application's namespaces.
+- Boots Rails (`config/environment.rb`) with
+  `unshareable_proc_action` armed, so every callback block Rails
+  cannot make shareable is reported at the Proc's definition site;
+  eager-loads; on Rails 8.2 calls `ractorize!` to freeze the
+  application graph, then serves one GET / on the main Ractor
+  (where a lazy memo on a now-frozen object raises FrozenError)
+  and one inside a Ractor; and sweeps the application's
+  namespaces. Without `ractorize!` an info note says so.
 
 ## Agent skill
 
 This repository ships a `ractor-readiness` skill that teaches
-coding agents (Claude Code and friends) the full audition
+coding agents (Claude Code and friends) the full Audition
 workflow: audit, fix tiers, suite-parity verification, and
 incremental adoption. It lives in
 [skills/ractor-readiness/SKILL.md](skills/ractor-readiness/SKILL.md).
@@ -402,7 +478,7 @@ bundle exec exe/audition --capabilities
 ```
 
 Static scanning is Ractor-parallel on large targets (one worker
-per core, minus one for the main Ractor); audition's own `lib/`
+per core, minus one for the main Ractor); Audition's own `lib/`
 passes `audition lib` clean.
 
 The design notes in `docs/design.md` include the empirically
@@ -413,7 +489,7 @@ calibrated against.
 
 The whole-program checks stand on
 [rubydex](https://github.com/Shopify/rubydex), Shopify's
-high-performance static analysis suite for Ruby: audition feeds
+high-performance static analysis suite for Ruby: Audition feeds
 every file into its graph and reads state ownership back out.
 Thanks to its authors, in particular the top five contributors:
 [Alexandre Terrasa](https://github.com/Morriar),

data/lib/audition/bundle_sweep.rb

@@ -31,10 +31,10 @@ module Audition
 
     # Audits every locked gem and ranks the results, worst first.
     #
-    # @param progress [Proc, nil] called with (row, done, total)
-    #   as each gem finishes
+    # @param progress [Progress] named per gem as each finishes,
+    #   which is the only unit a sweep has
     # @return [Array<Row>]
-    def rows(progress: nil)
+    def rows(progress: Progress::SILENT)
       gems = locked_gems
       queue = Thread::Queue.new
       gems.each { |g| queue << g }
@@ -42,17 +42,19 @@ module Audition
 
       collected = []
       mutex = Thread::Mutex.new
-      @concurrency.times.map do
-        Thread.new do
-          while (name, version = queue.pop)
-            row = audit_gem(name, version)
-            mutex.synchronize do
-              collected << row
-              progress&.call(row, collected.size, gems.size)
+      progress.phase("sweep", total: gems.size, unit: "gems") do
+        @concurrency.times.map do
+          Thread.new do
+            while (name, version = queue.pop)
+              row = audit_gem(name, version)
+              mutex.synchronize do
+                collected << row
+                progress.item(row.name)
+              end
             end
           end
-        end
-      end.each(&:join)
+        end.each(&:join)
+      end
 
       collected.sort_by do |row|
         [VERDICT_ORDER.fetch(row.verdict), -row.errors, row.name]
@@ -78,6 +80,7 @@ module Audition
       findings = filter(static_findings(target, config),
         directives, config)
       results = dynamic_results(target)
+      findings = Reconciliation.apply(findings, results)
       findings += filter(results.flat_map(&:findings),
         directives, config)
       report = Report.new(
@@ -115,8 +118,10 @@ module Audition
         ruby_files: spec.require_paths.flat_map do |rp|
           ruby_files_under(File.join(root, rp))
         end,
-        entry: {mode: :require, feature: spec.name, root: root},
-        compiled_files: Target.compiled_for(spec)
+        entry: {mode: :require, feature: spec.name,
+                load_paths: spec.full_require_paths, root: root},
+        compiled_files: Target.compiled_for(spec),
+        stub_files: Target.stubs(root)
       )
     end
 
@@ -148,7 +153,12 @@ module Audition
       end
       per_file = Static::Analyzer.new
         .analyze_paths(files, workers: 1)
-      per_file + Static::GraphAudit.new.analyze_paths(files) +
+      gem_calls = Static::GemCalls
+        .new(root: target.root, stubs: target.stub_files)
+        .analyze_paths(files)
+      per_file + gem_calls + Static::GraphAudit.new
+        .analyze_paths(files, constant_findings: per_file + gem_calls,
+          workers: 1) +
         Static::NativeExtensions.new.analyze(target,
           compiled_files: compiled)
     end