pinspec 0.2.0 → 0.3.0

checksums.yaml

@@ -1,7 +1,7 @@
 ---
 SHA256:
-  metadata.gz: 3725fabfaf5ca20380e8006be8da9324aa2e1f044dba71ee7cd867360a42fe57
-  data.tar.gz: 4d83085aaa24d6b1eb97577f32fdc8bb4696c28469bacfab8187c04cd12f3e5a
+  metadata.gz: 18f3a4d549580bfcb2be37ffa3636d014b22d882e34c969454088406f3e9f162
+  data.tar.gz: 7c3079d9f32f69e8dfc62cf47374df80029cb0423a1662f98edf628c8c1f9e50
 SHA512:
-  metadata.gz: 62faf9d5c24304fe2d122d05ddf17634185728764e4a59dd5bdbba8c953e718d748497abd562d95e77ed35c8dc4b1477a1bc0732fb403498f846498f3572cbd3
-  data.tar.gz: 5af0cf46d7730057402c1b3dcc0679ec495be239b94fba0aecae7e21a1029fdedfa15f053404f7f52e5594e9706522820cac109fe68abf9de24f7da323f9a121
+  metadata.gz: f9371ec4351d1b66131026e04b847ca7bdc363006de97de5a8486d8aede22e29461ed11dac2ad69202a70b393e5ea1634c6ff63b33e24ed6cc8545afd0b52ed6
+  data.tar.gz: fdb55aad3d613a22b7d2956197ec916b9cdb0a93d59b1aa84a2620e4f0e00eaec4b64674bb178b553285fe0379ffbe8a75e00b510af0bd138d993da2c825bd17

data/CHANGELOG.md

@@ -1,5 +1,133 @@
 # Changelog
 
+## 0.3.0 - 2026-08-15
+
+More of a real codebase reachable, four things that were quietly wrong put right, and
+294 fewer lines of code.
+
+Measured across five public Rails applications (openfoodnetwork, chatwoot, mastodon,
+forem, publishing-api — 841 service files), the share pinspec can plan a world for:
+
+| | before | after |
+|---|---|---|
+| openfoodnetwork | 16% | 39% |
+| chatwoot | 2% | 45% |
+| mastodon | 10% | 76% |
+| forem | 23% | 57% |
+| publishing-api | 24% | 67% |
+| **all** | **14%** | **53%** |
+
+### Reaching more targets
+
+- **The target method is discovered, not assumed.** `#call` is a convention, not the
+  convention — chatwoot's entry points are named `perform`. pinspec counts the method
+  names a directory actually uses and follows that application's own convention,
+  falling back to `call`/`perform`/`run`/`execute`/`process`, then a class's only
+  public method. Several plausible methods and no conventional one is a question, not
+  a guess.
+- **`def self.call(...)` delegating to `def call`** is the commonest service idiom in
+  Ruby, and it made the bare name resolve to two definitions — 108 of forem's 322
+  files were refused for it. The instance method is now named explicitly.
+- **A class with no `initialize` has the default constructor, not an unreadable one.**
+  Superclasses are resolved across the application's own files; only a constructor
+  that exists and cannot be read is still refused. That one change took mastodon from
+  10% to 76%.
+- **Two type-hint rules.** In 222 of 222 parameters refused as unbuildable models, the
+  "type" was just the parameter's own name capitalised — the refusal was rejecting
+  pinspec's own guess. Names shaped like `create_params` or `options` are Hashes and
+  plural names are Arrays, which are values pinspec can actually build.
+
+### `pinspec verify SPEC_FILE`
+
+Runs **any** spec file in the three environments, whoever wrote it. The Verifier never
+knew where a spec came from, so this needed a command and nothing else. A spec
+asserting `Time.now.strftime("%z")` passes where it was written and fails under
+`hostile` — the failure a colleague's CI would have reported a week later.
+
+### Fixed
+
+- **`neighbored` never ran anything twice.** It passed the same path to RSpec twice,
+  and RSpec loads a given path once — so the configuration silently re-checked what
+  `isolated` checks, while the README claimed it caught accumulated state. It now runs
+  a copy alongside the original, which really does execute the pin twice. On its first
+  honest run it immediately found a real defect: **factory_bot sequences are
+  process-global**, so a pin whose world uses one produced `INV-1` then `INV-2` and
+  failed against its own snapshot. Both hosts now rewind sequences before every case.
+- **`--boots 1` silently defeated the central claim.** One run has nothing to compare
+  against, so every case was called stable by virtue of never being checked. Two is
+  now a floor, not a default.
+- **The emitted spec dropped a factory record's attributes and associations.** The
+  probe passed them; the spec did not, so the two hosts built different worlds.
+- **`--sample` bound to the factory record, not the sampled row.** Imports are now
+  built first, so the real row the flag exists to fetch is the one the target receives.
+- **`redact:` in `.pinspec.yml` was accepted and ignored.**
+
+### Two constructor shapes real applications use
+
+- **attr_extras.** `pattr_initialize`/`attr_initialize` generate the constructor, so
+  there is no `def initialize` to read and pinspec concluded the class took no
+  arguments - then called `new` with none, and the generated constructor raised
+  KeyError inside the probe. That is **110 of chatwoot's 386 service files**. Both
+  forms are now read: bare symbols are positional, symbols inside an array are
+  keywords, and attr_extras' trailing `!` marks the required ones.
+- **A module that answers its own methods.** `module_function` and `extend self` mean
+  there is nothing to construct, but pinspec called `.new` on the module and the probe
+  died with `undefined method 'new' for module ...` - an error naming the application
+  rather than the shape pinspec had failed to recognise.
+
+Both were found by booting real applications rather than by reading fixtures.
+
+One consequence is worth stating plainly: chatwoot's plannable share **fell** from
+109 files to 97. Those twelve were planning successfully only because pinspec could
+not see their constructor at all; they would have failed inside the probe. Refusing
+them up front, with the parameter named, is the honest answer - `channel` alone
+accounts for 12, and it is genuinely unbuildable because chatwoot's channels are
+separate models with separate tables.
+
+### Upgrading from 0.1.0 or 0.2.0
+
+Nothing you already have stops working. Verified by generating pins with the real
+0.1.0 and 0.2.0 and putting them through this version:
+
+- **Pins written by 0.1.0 and 0.2.0 still verify green**, unchanged, including under
+  the new honest `neighbored`.
+- **Re-pinning one keeps its method and its filename.** Discovery follows the
+  application's convention, which can differ from the `#call` older versions always
+  assumed; where a pin already exists, the method it froze wins, so re-running cannot
+  quietly write a second file and leave the first one in the suite.
+- **Every flag those versions accepted is still accepted**, including `--skip-verify`
+  and the `--app-env A=1 B=2` array form. `--snapshot` is taken and ignored with a
+  note rather than rejected - it is no longer advertised, because a flag that does
+  nothing should not invite use.
+- **A `.pinspec.yml` naming a retired key warns and continues** instead of failing the
+  build. A key that was never valid is still an error.
+- **Exit codes mean what they always meant.** 11 (`EnvironmentRefused`) is retired and
+  deliberately not reused, so a script testing for it simply never sees it.
+
+`spec/upgrade_compatibility_spec.rb` holds all of this, so it cannot regress quietly.
+
+### Deleted
+
+294 lines net. Nothing here changes behaviour any user could observe:
+
+- `Emit::Namer` (253 lines with its spec) — an LLM-backed description generator with
+  an injectable client port, a prompt builder and an output sanitiser, reachable from
+  nothing. There was no flag that turned it on.
+- `--snapshot` — a flag whose entire implementation refused two of its own three
+  values. Removed rather than given an abstraction to hang implementations on.
+- `Sampler.choose_env` / `production_like?` / `guard_production!` and
+  `EnvironmentRefused` — a guard for a path with no caller.
+- `FactoryIndex#ancestry` / `#traits_for` / `#attributes_for` — one concept
+  implemented twice; this was the copy nothing called.
+- `Sandbox`'s `timeout:` and `runner:` seams (there is no timeout wrapper anywhere),
+  `MAX_GENERATIONS`, and 31 of 41 `NON_MODEL_HINTS` entries that fire zero times.
+
+### Added
+
+- A **vacuous-pin caveat** in the report: a case that returned nothing, wrote no rows,
+  enqueued nothing and sent nothing did run, but almost any change to the target would
+  still satisfy it. These are exactly the pins that score weak.
+
 ## 0.2.0 - 2026-08-15
 
 Ergonomics. 0.1.0 worked, but a run against a real application needed six lines of

data/README.md

@@ -63,6 +63,7 @@ on the command line always wins.
 | Command | What it does |
 | --- | --- |
 | `pinspec pin TARGET` | Capture, emit the spec, verify it. This is the one you want. |
+| `pinspec verify SPEC_FILE` | Run **any** spec file in the three environments — including one a human or an agent wrote. |
 | `pinspec init` | Write `.pinspec.yml` so later runs need no flags. |
 | `pinspec analyze` | App profile, schema, factories and hazards. Reads files only — no boot, no database. |
 | `pinspec validate TARGET` | Mutation-scores the pin, one aspect at a time. Needs Ruby >= 3.4 and `mutineer`. |
@@ -73,11 +74,18 @@ on the command line always wins.
 `TARGET` is a file, a `FILE#METHOD`, or a directory:
 
 ```bash
-pinspec pin app/services/invoice_calculator.rb          # assumes #call
-pinspec pin app/services/invoice_calculator.rb#total    # a different method
+pinspec pin app/services/invoice_calculator.rb          # discovers the method
+pinspec pin app/services/invoice_calculator.rb#total    # or name it yourself
 pinspec pin app/services                                # everything under it
 ```
 
+When you do not name a method, pinspec finds one: it counts the method names the
+directory actually uses and follows that convention, so an application whose entry
+points are `perform` needs no configuration. Failing that it looks for
+`call`, `perform`, `run`, `execute`, `process`, then a class's only public method.
+When several public methods are plausible and none is conventional it **asks**
+rather than picking, listing what it found. `--method NAME` settles it.
+
 Pinning a directory keeps going when a target is refused, and prints one summary:
 
 ```
@@ -102,13 +110,26 @@ captured it proves repeatability rather than portability:
 
 - **isolated** — the file alone, as captured.
 - **hostile** — a different timezone, locale and RSpec seed.
-- **neighbored** — the file twice in one process, so accumulated state shows up.
+- **neighbored** — a copy of the file alongside it, so state left behind by one run shows up in the next.
 
 The emitted spec forces the capture's answer on every axis rather than inheriting
 the suite's: isolation regime, queue adapter, clock, seed, locale and zone. A suite
 that truncates instead of transacting, or that runs jobs inline, would otherwise
 turn a green capture into a red or vacuous spec.
 
+## Verifying specs pinspec did not write
+
+Everyone has an agent that writes tests now. Almost nobody runs them anywhere but the
+machine that wrote them.
+
+```bash
+pinspec verify spec/models/order_spec.rb
+```
+
+Same three environments, on any RSpec file. A spec asserting `Time.now.strftime("%z")`
+passes where it was written and fails under `hostile` — which is what a colleague's CI
+in another timezone would have told you a week later.
+
 ## No database ids in a pin
 
 Postgres sequences are not transactional, so a rolled-back case still advances them

data/lib/pinspec/analyzer/discovery.rb

@@ -0,0 +1,162 @@
+# frozen_string_literal: true
+
+require "prism"
+
+module Pinspec
+  module Analyzer
+    class Discovery
+      CONVENTIONAL = %w[call perform run execute process].freeze
+
+      # Never a target: these are object protocol, not behaviour anyone pins.
+      NON_TARGETS = %w[initialize to_s to_str inspect hash eql? == <=> to_proc].freeze
+
+      # Conversion methods, which are USUALLY protocol but are sometimes the whole
+      # public surface of a service object - OFN's AvailablePaymentMethodsService
+      # exposes exactly `to_a` and nothing else. Excluding them outright meant such a
+      # class had no candidates at all and was refused as ambiguous. They are ranked
+      # last instead, so they win only when nothing else is offered.
+      LAST_RESORT = %w[to_a to_h each].freeze
+
+      Choice = Data.define(:method_name, :reason, :candidates, :owner) do
+        def ambiguous?
+          method_name.nil?
+        end
+
+        # `def self.call(...)` delegating to `def call` is the commonest service-object
+        # idiom in Ruby, and it makes the bare name resolve to two definitions. The
+        # instance method is the real entry point, so it is named explicitly rather
+        # than left to look ambiguous.
+        def descriptor
+          owner ? "#{owner}##{method_name}" : method_name
+        end
+      end
+
+      # `convention` is the method name this application uses most, counted once over
+      # the directory being pinned. Hardcoding `call` reads 2% of a codebase whose
+      # services are named `perform`.
+      def self.convention_for(files)
+        counts = Hash.new(0)
+
+        files.each do |file|
+          surface = new(file).surface
+          surface[:instance].each { |name| counts[name] += 1 }
+        end
+
+        best = counts.max_by { |name, count| [count, CONVENTIONAL.index(name) ? 1 : 0] }
+        return nil if best.nil? || best.last < 2
+
+        best.first
+      end
+
+      def initialize(file_path)
+        @file_path = file_path
+      end
+
+      def surface
+        @surface ||= read_surface
+      end
+
+      def choose(convention: nil)
+        instance = surface[:instance]
+        singleton = surface[:singleton]
+
+        if instance.empty? && singleton.empty?
+          return Choice.new(method_name: nil, reason: :no_public_methods, candidates: [], owner: nil)
+        end
+
+        preferred = [convention].compact + CONVENTIONAL
+
+        preferred.each do |name|
+          return chosen(name, reason_for(name, convention), instance) if instance.include?(name)
+        end
+
+        preferred.each do |name|
+          return Choice.new(method_name: name, reason: :class_method, candidates: singleton, owner: nil) if singleton.include?(name)
+        end
+
+        # A class with exactly one public method has only one thing it can mean.
+        return chosen(instance.first, :sole_method, instance) if instance.size == 1
+
+        # Nothing conventional, and what remains is a single conversion method: that is
+        # this class's whole public surface, so it is the target.
+        # Conversion methods rank last, so a real entry point wins - but a class whose
+        # only ordinary method is one of them still has something to pin.
+        ordinary = instance - LAST_RESORT
+        return chosen(ordinary.first, :sole_method, instance) if ordinary.size == 1
+        return Choice.new(method_name: singleton.first, reason: :sole_method, candidates: singleton, owner: nil) if instance.empty? && singleton.size == 1
+
+        Choice.new(method_name: nil, reason: :ambiguous, candidates: (instance + singleton).first(8), owner: nil)
+      end
+
+      private
+
+      # Qualified only when the same name also exists as a class method, which is
+      # what makes the bare name look like two definitions.
+      def chosen(name, reason, candidates)
+        doubled = surface[:singleton].include?(name)
+
+        Choice.new(method_name: name, reason: reason, candidates: candidates,
+                   owner: doubled ? surface[:owners][name] : nil)
+      end
+
+      def reason_for(name, convention)
+        name == convention ? :convention : :conventional_name
+      end
+
+      def read_surface
+        source = Source.read(@file_path)
+        result = Prism.parse(source)
+        return { instance: [], singleton: [], owners: {} } unless result.success?
+
+        instance = []
+        singleton = []
+        owners = {}
+        visibility = :public
+        class_stack = []
+
+        walk = lambda do |node, in_singleton|
+          return if node.nil?
+
+          case node
+          when Prism::CallNode
+            # `private` on its own switches visibility; `private :foo` names one method.
+            visibility = node.name if %i[private protected public].include?(node.name) && node.arguments.nil?
+          when Prism::ClassNode
+            visibility = :public
+            class_stack.push(node.constant_path.slice)
+            node.compact_child_nodes.each { |child| walk.call(child, in_singleton) }
+            class_stack.pop
+            return
+          when Prism::ModuleNode
+            visibility = :public
+          when Prism::SingletonClassNode
+            node.compact_child_nodes.each { |child| walk.call(child, true) }
+            return
+          when Prism::DefNode
+            collect(node, in_singleton, visibility, instance, singleton)
+            owners[node.name.to_s] ||= class_stack.last unless node.receiver || in_singleton
+          end
+
+          node.compact_child_nodes.each { |child| walk.call(child, in_singleton) }
+        end
+
+        walk.call(result.value, false)
+
+        { instance: instance.uniq, singleton: singleton.uniq, owners: owners }
+      end
+
+      def collect(node, in_singleton, visibility, instance, singleton)
+        name = node.name.to_s
+        return unless visibility == :public
+        return if NON_TARGETS.include?(name)
+        return if name.end_with?("=")
+
+        if node.receiver || in_singleton
+          singleton << name
+        else
+          instance << name
+        end
+      end
+    end
+  end
+end

data/lib/pinspec/analyzer/factory_registry.rb

@@ -200,7 +200,7 @@ module Pinspec
           name = decode(args.first)&.to_sym
           return nil unless name
 
-          attribute(name, :association, node, factory: keyword_options(args)[:factory]&.to_sym, source: nil)
+          attribute(name, :association, node, factory: factory_name(args), source: nil)
         when :sequence
           name = decode(args.first)&.to_sym
           return nil unless name
@@ -216,17 +216,26 @@ module Pinspec
         return nil if BARE_NON_ATTRIBUTES.include?(node.name)
         return nil if STRUCTURAL_CALLS.include?(node.name)
 
-        if node.block
+        if node.block.is_a?(Prism::BlockNode)
           attribute(node.name, :block, node)
         elsif args.empty?
           attribute(node.name, :association, node)
         elsif args.all? { |arg| arg.is_a?(Prism::KeywordHashNode) }
-          attribute(node.name, :association, node, factory: keyword_options(args)[:factory]&.to_sym)
+          attribute(node.name, :association, node, factory: factory_name(args))
         else
           attribute(node.name, :static, node)
         end
       end
 
+      # factory_bot accepts an array here - `factory: %i[user admin]` names a factory
+      # and then traits to apply to it - so the value is not always a symbol.
+      def factory_name(args)
+        declared = keyword_options(args)[:factory]
+        declared = declared.first if declared.is_a?(Array)
+
+        declared&.to_sym
+      end
+
       def attribute(name, kind, node, factory: nil, source: :from_node)
         FactoryAttribute.new(
           name:    name.to_sym,
@@ -238,7 +247,7 @@ module Pinspec
       end
 
       def attribute_source(node)
-        if node.block
+        if node.block.is_a?(Prism::BlockNode)
           node.block.body&.slice
         else
           args = Array(node.arguments&.arguments).reject { |a| a.is_a?(Prism::KeywordHashNode) }