specguard-ruby 0.3.1

data/lib/specguard/rspec/annotation_lookup.rb

@@ -0,0 +1,363 @@
+# frozen_string_literal: true
+
+# Answers one question, for one example at a time: *is this test annotated, and
+# with what?*
+#
+# It is the formatter's half of the annotation story. The linter's half already
+# exists and is not duplicated here — {SpecGuard::RSpec::Scanner} finds every
+# `@intent:` in a file, captures its payload string-aware and parses it, and
+# the `validate-intent` binary decides whether the result is valid. This class
+# consumes both. Writing a second extractor would guarantee that the tool
+# telling an author their annotation is wrong and the tool reporting it to the
+# platform eventually disagree about what an annotation *is*.
+#
+# Requiring the linter's chain from here is safe in the direction that matters:
+# `lib/specguard/rspec.rb` does not require `rspec/core`, so pulling it in from
+# the formatter's side keeps `bin/specguard-lint` loadable on a machine with no
+# RSpec installed. spec/specguard/rspec/formatter_loading_spec.rb pins that in
+# both directions.
+require_relative "../rspec"
+
+module SpecGuard
+  module RSpec
+    # == The lookback rule (SPGD-12 §2)
+    #
+    # `example.metadata[:line_number]` is the line the `it` is on. An annotation
+    # counts as this example's when it sits on that line (the trailing form) or
+    # on the line immediately above it (the preceding-comment form):
+    #
+    #   # @intent: { entity: "Order", ... }    <- line above
+    #   it "restores stock on refund" do       <- metadata[:line_number]
+    #
+    #   it "surfaces the decline reason" do # @intent: { ... }   <- same line
+    #
+    # One line of lookback, no more. Multi-line annotations are unsupported in
+    # v1 (PROTOCOL.md §1 requires an annotation to fit on one line), so a wider
+    # window could only ever attach an annotation to the wrong example.
+    #
+    # == Only the comment form is inherited by the line below
+    #
+    # The two forms are not symmetric, and the difference is not cosmetic. A
+    # comment-only line hosts no example, so an annotation written there has
+    # exactly one possible claimant — the line under it. An annotation written
+    # in the trailing form is on a line that *is* an example's, so it has two,
+    # and one-liner specs make the second one ordinary rather than exotic:
+    #
+    #   it { is_expected.to eq(1) } # @intent: { entity: "Order", ... }
+    #   it { is_expected.to be_positive }
+    #
+    # Lookback that did not distinguish the forms would report **both** of those
+    # as annotated, carrying the same intent — so the platform would store the
+    # second example's declared purpose as something nobody declared, and count
+    # it in `annotated_specs_count`. That is worse than reporting it
+    # unannotated: a missing annotation is a visible gap, an invented one is
+    # confident false telemetry, and it inflates the very ratio `status` exists
+    # to keep honest.
+    #
+    # So a Finding is claimable by the line below it only when its own line is
+    # comment-only ({COMMENT_LINE}). Deciding that needs the source line, which
+    # is why {#build_index} reads the file itself and calls {Scanner.scan_text}
+    # rather than {Scanner.scan_file} — the read is the same single read either
+    # way, and the alternative was a second one per file purely to re-derive
+    # what the first one already had in hand.
+    #
+    # == Why a malformed annotation is reported as *unannotated*
+    #
+    # {AnnotationScanner} states, correctly, that a typo'd annotation "must fail
+    # loudly rather than silently counting as unannotated". That is the
+    # **linter's** stance, and the linter already acts on it: it prints the
+    # file:line and exits 1. It cannot bind the formatter, which is under the
+    # never-block-CI contract and has no way to be loud that is not also a way
+    # to be in someone's build output.
+    #
+    # The alternative is worse than it looks. `Ingest::Payload` is
+    # all-or-nothing — it collects errors globally and `valid?` requires the
+    # list to be empty — so shipping one schema-invalid intent does not lose
+    # that annotation, it loses **the whole run**: a 400, and no telemetry for
+    # any of the other twenty thousand examples. Downgrading one annotation to
+    # `unannotated` costs one row's metadata. The example's name, duration and
+    # outcome still ship either way.
+    #
+    # So every one of these produces the same answer — `nil`, meaning
+    # unannotated:
+    #
+    #   * no `@intent:` on either candidate line;
+    #   * an `@intent:` on the line above that is *not* a comment-only line —
+    #     the trailing form belongs to its own example and lends itself to
+    #     nobody;
+    #   * an `@intent:` whose payload could not be captured or parsed
+    #     ({Finding#problem?} — `KIND_EXTRACTION` / `KIND_PARSE`);
+    #   * a file that could not be read at all (`KIND_READ`);
+    #   * a syntactically fine annotation the schema rejects;
+    #   * the schema itself failing to load.
+    #
+    # == Cost
+    #
+    # {AnnotationScanner.each_intent} walks every line of the file it is handed.
+    # Scanning per *example* would make a 20,000-example suite read and walk its
+    # spec files 20,000 times, on the critical path of somebody's test run. So
+    # each file is read and scanned at most once and reduced to an {Index}:
+    # O(files), not O(examples). Schema validation is memoized on the same
+    # principle — per distinct annotation, not per example carrying it.
+    class AnnotationLookup
+      # Findings at line 0 are {Finding::KIND_READ} — "this file was never
+      # read", not "there is an annotation on line 0". Keeping the sentinel out
+      # of the index is what makes {Index#finding_for}'s unconditional
+      # `line - 1` lookback safe for an example on line 1.
+      FIRST_REAL_LINE = 1
+
+      # A line whose only content is a comment: the preceding-comment form, and
+      # the only form an example on the next line may claim. Leading whitespace
+      # is allowed because annotations are indented with the examples they
+      # describe.
+      COMMENT_LINE = /\A\s*#/
+
+      # One file's annotations, split by who is allowed to claim them.
+      #
+      #   own          every annotation, by the line it was written on
+      #   inheritable  only those in the comment form, by that same line
+      #
+      # The values are the INTENT TO SHIP or nil — the verdict already applied —
+      # rather than a Finding, because the two paths that build this index reach
+      # a verdict by different routes (the local Scanner for the LINES, or the
+      # port's report for the verdicts) and only agree on the answer. Storing the answer is what
+      # keeps `#intent_for` from having to know which one ran.
+      #
+      # A line's own annotation always wins: the trailing form is written *on*
+      # the example, so when a file somehow carries both it is the more specific
+      # of the two, and a malformed trailing annotation is not quietly
+      # backfilled from the comment above it — that would attach an intent its
+      # author had already stopped meaning. Hence `key?` rather than a truthy
+      # test: a line whose own annotation was rejected answers nil, and must not
+      # fall through to the comment above it.
+      #
+      # `line - 1` is 0 for an example on line 1, and {AnnotationLookup#build_index}
+      # is what guarantees that reads back as "no annotation": line 0 is never a
+      # key, because the only finding produced there is the KIND_READ sentinel.
+      Index = Struct.new(:own, :inheritable) do
+        # @return [Hash, nil]
+        def intent_for(line)
+          return own[line] if own.key?(line)
+
+          inheritable[line - 1]
+        end
+      end
+
+      # The answer for a file nothing could be read out of. Its own constant so
+      # the pessimistic pre-scan cache in {#index_for} and the unreadable-file
+      # path cannot drift apart.
+      EMPTY_INDEX = Index.new({}.freeze, {}.freeze).freeze
+
+      # @param env [Hash, ENV] where `SPECGUARD_VALIDATE_INTENT` is read from.
+      #   Injected for testing, and read LAZILY — see {#backend}.
+      def initialize(env: ENV)
+        @env = env
+        @indexes = {}
+      end
+
+      # The intent to attach to one example, or nil when it is unannotated.
+      #
+      # Nothing here is rescued: a caller that cannot survive an exception must
+      # say so itself, and the formatter does — it wraps this in the same
+      # `never_fail_the_run` envelope as everything else, so a blow-up costs the
+      # example its annotation and not the suite its exit code. The memoization
+      # below is deliberately written so that a failure is cached too, which is
+      # what keeps "warns once" from becoming "warns once but rescans the file
+      # for every one of the remaining examples".
+      #
+      # @param file [String, nil] the example's file, as the payload records it
+      # @param line [Integer, nil] `example.metadata[:line_number]`
+      # @return [Hash, nil] the parsed, schema-valid annotation
+      def intent_for(file:, line:)
+        return nil unless file.is_a?(String) && !file.empty?
+        return nil unless line.is_a?(Integer) && line >= FIRST_REAL_LINE
+
+        index_for(file).intent_for(line)
+      end
+
+      private
+
+      # The validator backend, or nil — memoized, including when resolving
+      # RAISES.
+      #
+      # WHY THIS CLASS RESOLVES ONE AT ALL. `CLI` routes the linter through the
+      # backend, and since the SPGD-867 cutover the backend is the ONLY
+      # validator: the gem no longer carries a Ruby validation arm. Whatever
+      # the backend accepts is what the platform must receive, so this asks
+      # the same binary the same question rather than a second parser that
+      # could disagree with it.
+      #
+      # Lazily, not in the constructor: {Formatter} builds this at suite start,
+      # and `resolve` verifies the binary (two probes, and a raise when it is
+      # unusable). Doing that during construction would move a failure out of
+      # the formatter's `never_fail_the_run` envelope and into the point where
+      # RSpec is still wiring itself up.
+      #
+      # A {ValidatorError} HERE IS NOT FATAL AND IS NOT nil-THE-ANNOTATION. An
+      # unusable binary means this class cannot ask the backend what an
+      # annotation says; it does NOT mean the annotation is bad. Since the
+      # cutover there is no Ruby arm to fall back to, so probe failure is
+      # memoized as nil and `#verdicts_for` answers `unverified_verdicts` for
+      # every file: local line discovery keeps working and every annotation is
+      # reported as `unannotated` — the honest answer under the formatter's
+      # never-fail-the-run contract, while the linter half makes the failure
+      # loud (it exits 2 when no binary can be resolved, naming the
+      # remediations).
+      #
+      # The alternative was letting the raise reach {Formatter}'s
+      # `never_fail_the_run` envelope. That envelope keeps the RUN alive, but
+      # it does so by abandoning the example's annotation. Degrading HERE
+      # instead means ONE swallowed probe per suite rather than one per
+      # example, with `unverified_verdicts` as the single consistent answer
+      # for every file.
+      #
+      # Cached, including on failure, for the reason {#schema} is: the probe is
+      # a subprocess, and re-running it per example is the O(examples) cost this
+      # class exists to refuse — on exactly the unhappy path where somebody's CI
+      # is already having a bad day.
+      def backend
+        return @backend if defined?(@backend)
+
+        @backend = nil
+        @backend = ValidatorBackend.resolve(env: @env)
+      rescue ValidatorError
+        # Deliberately swallowed rather than re-raised: @backend is already nil,
+        # so #verdicts_for takes the unverified_verdicts arm and the memo keeps
+        # this from being re-probed.
+        @backend = nil
+      end
+
+      # @return [Index]
+      def index_for(file)
+        return @indexes[file] if @indexes.key?(file)
+
+        # Cache the pessimistic answer *before* the scan, not after. If the scan
+        # raises, the raise still reaches the formatter (which warns once and
+        # moves on) but the empty index stays behind, so the next example in the
+        # same file is answered from the cache instead of re-raising and
+        # re-reading. Without this, "warns once" would still mean "re-reads a
+        # broken file once per example" — the O(examples) cost this class exists
+        # to remove, reappearing on exactly the unhappy path where somebody's CI
+        # is already having a bad day.
+        @indexes[file] = EMPTY_INDEX
+        @indexes[file] = build_index(file)
+      end
+
+      # @return [Index]
+      def build_index(file)
+        # The read happens on BOTH paths and first on both. The backend does not
+        # need it to find annotations — it reads the file itself — but the
+        # comment-form rule below is a property of the LINE an annotation sits
+        # on, which no report carries, so the text is required either way. Doing
+        # it first also keeps the two paths agreeing about an unreadable file
+        # without a subprocess being started to rediscover it.
+        text = read(file)
+        return EMPTY_INDEX if text.nil?
+
+        index_from(verdicts_for(file, text), text)
+      end
+
+      # One shell-out per FILE, which is the cost model this class already
+      # commits to: {#index_for} is O(files), `--source` takes files, and the
+      # port reports every annotation in one pass with the `file:line` scoping
+      # {Index} needs. Per-example would be O(examples) subprocesses, which is
+      # the cost this class exists to refuse.
+      #
+      # `#check` returns one {Linter::Result} per finding. A result that is not
+      # `ok?` is an annotation the linter fails the build over, so it ships
+      # nothing. Read failures and no-matches arrive line-scoped to 0 and are
+      # filtered by {#index_from}.
+      #
+      # A {ValidatorError} here means the backend could not answer about THIS
+      # file's annotations. Since the cutover there is no Ruby arm to fall back
+      # to, the honest answer is the one this class already gives for anything
+      # it could not verify: every annotation in the file ships `unannotated`.
+      # The never-block-CI contract forbids anything louder from the formatter,
+      # and the linter — which CAN be loud — is the tool that catches it.
+      def verdicts_for(file, text)
+        resolved = backend
+        return unverified_verdicts(file, text) if resolved.nil?
+
+        resolved.check([file]).map do |result|
+          [result.line, result.ok? ? result.representable_intent : nil]
+        end
+      rescue ValidatorError
+        unverified_verdicts(file, text)
+      end
+
+      # The lines the scanner still finds, every one answered `nil`. Discovery
+      # is a property of the FILE (which lines carry annotations, which are
+      # comment-form) and stays local — it is validation that needed the
+      # backend. Keeping the lines lets {#index_from} apply the comment-form
+      # rule exactly as it does for verdicts the backend did answer.
+      def unverified_verdicts(file, text)
+        Scanner.scan_text(text, file: file).map { |finding| [finding.line, nil] }
+      end
+
+      # @return [Index]
+      def index_from(verdicts, text)
+        lines = nil
+        own = {}
+        inheritable = {}
+
+        verdicts.each do |line, intent|
+          # Findings at line 0 are {Finding::KIND_READ} — "this file was never
+          # read", not "there is an annotation on line 0".
+          next unless line.is_a?(Integer) && line >= FIRST_REAL_LINE
+
+          # First wins. `each_intent` resumes scanning after each captured
+          # payload, so trailing prose containing a second `@intent:` yields a
+          # second finding on the same line. The annotation is the one the
+          # author wrote first; the rest of the line is commentary. Stated here
+          # rather than left to Hash-insertion order.
+          next if own.key?(line)
+
+          own[line] = intent
+          # Materialized only once a real annotation has been found, and never
+          # for a file `scan_text` refused: its invalid-UTF-8 answer is a single
+          # line-0 finding, filtered out above, and splitting such a string is
+          # itself an ArgumentError waiting to happen.
+          lines ||= text.lines
+          inheritable[line] = intent if COMMENT_LINE.match?(lines[line - 1].to_s)
+        end
+
+        Index.new(own, inheritable)
+      end
+
+      # {Scanner.scan_file} would do this read, but it hands back only Findings
+      # and the form of an annotation is a property of the *line* it sits on. So
+      # the read happens here and {Scanner.scan_text} — the same pipeline, one
+      # stage lower — does the extracting. Nothing about what an annotation *is*
+      # is re-implemented; a second extractor is how the tool that tells an
+      # author their annotation is wrong and the tool that reports it to the
+      # platform end up disagreeing.
+      #
+      # The rescue mirrors `scan_file`'s own, and the two agree on the answer: it
+      # turns an unreadable file into a line-0 KIND_READ Finding, which this
+      # class filters out, and nil here becomes {EMPTY_INDEX}. Either way, every
+      # example in a file that could not be read is unannotated.
+      #
+      # @return [String, nil]
+      def read(file)
+        File.read(file, encoding: "UTF-8")
+      rescue SystemCallError, IOError
+        nil
+      end
+
+      # Keyed by the intent itself, so an annotation repeated across a suite is
+      # checked once rather than once per example that carries it. The
+      # shell-out is the expensive stage, and this class is explicitly about
+      # not paying O(examples) for work that is O(files).
+      #
+      # The verdict is now reached while the index is BUILT rather than when an
+      # example first claims the line, which makes the cost O(annotations in the
+      # file) instead of O(annotations examples ask about) — still bounded by
+      # the file, still memoized across files, and it is what lets the index
+      # hold one shape whichever path filled it. The visible consequence is that
+      # an unloadable schema raises on the first lookup into a file with any
+      # annotation rather than the first lookup that lands on one; both are
+      # inside the formatter's envelope, and {#index_for}'s pessimistic
+      # pre-caching already guarantees the second example does not raise again.
+    end
+  end
+end

data/lib/specguard/rspec/annotation_scanner.rb

@@ -0,0 +1,190 @@
+# frozen_string_literal: true
+
+module SpecGuard
+  module RSpec
+    # Finds `@intent:` annotations in test source and captures the object
+    # literal that follows each one.
+    #
+    # This is a *syntactic* pass only: it hands back the payload exactly as
+    # written (still in PROTOCOL.md §1's permissive syntax — see
+    # {PayloadNormalizer}) and never inspects its contents. Schema validation is
+    # a later stage entirely.
+    #
+    # The scan is **string-aware**: quoted content is skipped over wholesale, so
+    # a `behavior` sentence containing an apostrophe or an unbalanced `{` cannot
+    # terminate the capture early. Plain brace-counting gets that case wrong.
+    #
+    # The algorithm is PROTOCOL.md §1's: find each `@intent:` token and capture
+    # the object literal after it, string-aware and bracket-balanced.
+    # open-test-intent's `validate-intent` implements the same rules; where the
+    # two disagree, PROTOCOL.md decides.
+    module AnnotationScanner
+      INTENT_TOKEN = "@intent:"
+
+      # Closing bracket => the opener it must match.
+      OPENERS = { "}" => "{", "]" => "[" }.freeze
+
+      # The `{` at `start` was never closed on this line.
+      UNTERMINATED_OBJECT = "unterminated object literal (an annotation must fit on one line)"
+      UNBALANCED_BRACKETS = "unbalanced brackets in the annotation payload"
+      NO_PAYLOAD = "no '{...}' object literal follows the @intent: token"
+
+      module_function
+
+      # Yields `[line_no, raw_payload, problem]` for every `@intent:` token in
+      # `text`, in source order.
+      #
+      # `raw_payload` is the object literal as written; it is `nil` when
+      # `problem` explains why the annotation could not be captured.
+      #
+      # An `@intent:` token carrying no extractable payload is **reported, not
+      # skipped** — a typo'd annotation must fail loudly rather than silently
+      # counting as "this example is unannotated".
+      #
+      # @param text [String] the full source of one file
+      # @yieldparam line_no [Integer] 1-based line number
+      # @yieldparam raw_payload [String, nil]
+      # @yieldparam problem [String, nil]
+      # @return [Enumerator] when no block is given
+      def each_intent(text)
+        return enum_for(:each_intent, text) unless block_given?
+
+        text.each_line.with_index(1) do |raw_line, line_no|
+          line = raw_line.chomp
+          pos = 0
+
+          loop do
+            token_at = line.index(INTENT_TOKEN, pos)
+            break if token_at.nil?
+
+            brace_at = payload_brace(line, token_at)
+            if brace_at.nil?
+              yield line_no, nil, NO_PAYLOAD
+              break
+            end
+
+            begin
+              finish = scan_object(line, brace_at)
+            rescue ScanError => e
+              yield line_no, nil, e.message
+              break
+            end
+
+            yield line_no, line[brace_at...finish], nil
+
+            # Resume *after* the captured payload so trailing prose containing
+            # another `@intent:` is still seen, but the payload's own contents
+            # are never rescanned.
+            pos = finish
+          end
+        end
+      end
+
+      # Returns the index of the `{` opening the payload of the `@intent:`
+      # token at `token_at`, or nil when that token has no payload.
+      #
+      # The search is **bounded by the next `@intent:` token on the line**. An
+      # unbounded search reads to end of line, so a malformed token followed by
+      # a well-formed one adopts its neighbour's object literal and is yielded
+      # with `problem: nil` — a typo'd annotation reported as valid, carrying
+      # somebody else's intent, and (because the caller resumes past the
+      # captured payload) swallowing the well-formed token on the way. That is
+      # the exact opposite of {each_intent}'s "reported, not skipped" contract,
+      # and PROTOCOL.md §1 supports the bounded reading: a payload is the object
+      # *following* its own token, which a literal on the far side of a second
+      # token is not.
+      #
+      # The bound can only ever *shrink* the search — it never picks a
+      # different brace, only declines one — so a token whose payload lies
+      # beyond the next token takes the {NO_PAYLOAD} path instead of a false
+      # pass.
+      #
+      # `line.index` is a naive string search, so it also finds an `@intent:`
+      # written inside a quoted string. That is harmless for the case it looks
+      # like it would break — a token quoted *inside a payload* is by definition
+      # after that payload's `{`, so the bound is inert and the payload is
+      # captured as before. The occurrence has to sit between the token and its
+      # `{` to matter at all, which means it is in the prose separating them:
+      #
+      #   # @intent: like the "@intent:" above { entity: "Order", ... }
+      #
+      # That line now reports NO_PAYLOAD where it previously captured the
+      # literal. It is the one shape this bound makes stricter, and the
+      # stricter answer is the right one: the line is genuinely ambiguous about
+      # which token owns the literal, and declining to guess is the loud answer
+      # this scanner is supposed to give. Moving the quoted mention after the
+      # payload, or dropping the quotes, restores the capture.
+      def payload_brace(line, token_at)
+        after_token = token_at + INTENT_TOKEN.length
+        brace_at = line.index("{", after_token)
+        return nil if brace_at.nil?
+
+        next_token = line.index(INTENT_TOKEN, after_token)
+        return nil if next_token && brace_at > next_token
+
+        brace_at
+      end
+
+      # Returns the index just past the `}` matching the `{` at `text[start]`.
+      #
+      # Bracket-balanced and string-aware, so a brace inside a quoted value does
+      # not end the payload early. Annotations are single-line per PROTOCOL.md
+      # §1, so `text` is one line.
+      #
+      # @raise [ScanError] when the literal is unterminated or unbalanced
+      def scan_object(text, start)
+        stack = []
+        i = start
+        length = text.length
+
+        while i < length
+          char = text[i]
+
+          if char == '"' || char == "'"
+            i = scan_string(text, i, char)
+            next
+          end
+
+          if char == "{" || char == "["
+            stack.push(char)
+          elsif char == "}" || char == "]"
+            raise ScanError, UNBALANCED_BRACKETS if stack.empty? || stack.last != OPENERS[char]
+
+            stack.pop
+            return i + 1 if stack.empty?
+          end
+
+          i += 1
+        end
+
+        raise ScanError, UNTERMINATED_OBJECT
+      end
+
+      # Returns the index just past the closing `quote` of a string literal.
+      #
+      # `text[start]` must be the opening quote. Backslash escapes are honoured,
+      # so a quote or brace *inside* the string never terminates the scan.
+      #
+      # @raise [ScanError] when the string literal is unterminated
+      def scan_string(text, start, quote)
+        i = start + 1
+        length = text.length
+
+        while i < length
+          char = text[i]
+
+          if char == "\\"
+            i += 2 # skip the escape and whatever it escapes
+            next
+          end
+
+          return i + 1 if char == quote
+
+          i += 1
+        end
+
+        raise ScanError, "unterminated #{quote}-quoted string"
+      end
+    end
+  end
+end