condition_coverage 0.0.5 → 0.0.6

checksums.yaml

@@ -1,7 +1,7 @@
 ---
 SHA256:
-  metadata.gz: a61c323a92442b231f61a10ea7ee622ee5e3b957f9818819e56aefda2ed75efd
-  data.tar.gz: eba0b9b94f2ba0756f0549d9675a4d3cb4c548df4854114df6e76e7ce09cdbeb
+  metadata.gz: f113d970620b81b385c0ac7f1ce0e89251574bf8f14498be64b460decb35d315
+  data.tar.gz: 5f081efffd90f7d0c216d9e5cfbdb9746e2d1a4dee2384a54ad5b5f8b31cbc1f
 SHA512:
-  metadata.gz: f84973aba8aa9323ba03323f5fabe1b99feda591d0973a715e27f792a61971c50fb87b90680db5245451f16a90180dcae7272a8bdc33fd25998429d63860cd45
-  data.tar.gz: b992c871ed4f1bd658d828fec3da583199a4ddf3a44c26f0cb79b142f57f74e294a5f9c50e672780fb4f213986163f9c5b0a64e03903e462aebaf8c7a78e07ae
+  metadata.gz: 934738b27336db9ff08212d39601f1a8b64c99bcf2a989ff07369334ecdde3bf8d817963c9800ac010351589203e04dc06be14738a3eae759eae7a61d736946d
+  data.tar.gz: c6b92d448eeb7b62623de4c8cc6b135b13e1bea82dcb6a62555f590331fd6e53d5992b0998171eeb145ca032ccfbbaf21e6513c1448ac3497a5ef88bd582d04b

data/README.md

@@ -43,13 +43,47 @@ end
 A misspelt setting or an unknown reporter is refused by name, with the list
 of the ones there are. Calling `start` again replaces the configuration.
 
+A run whose code cannot be changed, such as a script in a `docker build`
+step, can be covered from its environment alone:
+
+```sh
+RUBYOPT=-rcondition_coverage/auto
+CONDITION_COVERAGE_ROOT=/app                  # root
+CONDITION_COVERAGE_COVER=src/**/*.rb,lib/*.rb  # cover, comma-separated
+CONDITION_COVERAGE_DIR=/tmp/condition         # coverage_dir
+CONDITION_COVERAGE_COLLECTOR=http://host.docker.internal:4570/runs
+```
+
+This starts with `report :post`. With no `CONDITION_COVERAGE_COLLECTOR` it
+starts nothing and prints nothing, so an image can set `RUBYOPT` for every
+run and measure only those given a collector.
+
+The gem's `condition_coverage_collector` command is that collector. It
+saves each report POSTed to it as a numbered file, and once every run is
+done they render as one report:
+
+```sh
+condition_coverage_collector 4570 /tmp/condition/runs   # until stopped
+```
+
+```ruby
+runs = Dir['/tmp/condition/runs/*.json']
+ConditionCoverage::Render.from_json(config, runs)
+```
+
 ## What a run reports
 
 As the process exits:
 
 - `condition_coverage.json` is always written to `coverage_dir`: every site
   of every covered file, with each operand's true and false counts. It is
-  the base the other reporters read.
+  the base the other reporters read, so they can also be run later, in
+  another process, from it alone:
+  `ConditionCoverage::Render.from_json(config)`, where `config` is a
+  `ConditionCoverage::Configuration` built from the same settings block.
+  Given several runs' reports, `Render.from_json(config, paths)` adds
+  their counts together first, so the report shows only the gaps that
+  every run left.
 - `:gaps` prints one line per missing outcome, in manifest order:
 
 ```text
@@ -79,7 +113,9 @@ An operand whose form allows one outcome only needs only to be evaluated:
 its other outcome is impossible, not a gap. `Foo.new`, literals (`'s'`,
 `1`, `:k`, `[]`, `{}`, `true`), lambdas and constants (`LATEST`,
 `Config::DEFAULT`, since one holding nil or false is rare) can only be
-truthy; `nil` and `false` only falsy:
+truthy, as can `$stdout` and `$stderr`, which Ruby refuses to set to nil
+or false. Other globals, `$stdin` among them, can be either. `nil` and
+`false` can only be falsy:
 
 ```ruby
 @clock = options[:clock] || MonotonicClock.new  # operand 1: never evaluated, or no gap
@@ -97,6 +133,12 @@ truthy; `nil` and `false` only falsy:
   the top of the page lists every site as a link to its line, covered
   sites first, then those with gaps, and arrows beside each site with
   gaps jump to the previous and next one, wrapping round at either end.
+- `:post` sends the JSON report, as an HTTP POST, to the url set by
+  `collector 'http://...'`. It is for a run whose own files are thrown
+  away, such as a failing `docker build` step. It prints nothing, never
+  goes through an `http_proxy`, and gives up in silence on a collector
+  that cannot be reached or does not answer within a second, so the
+  run's output and exit status are unchanged.
 
 ## How it works
 

data/exe/condition_coverage_collector

@@ -0,0 +1,26 @@
+#!/usr/bin/env ruby
+require 'fileutils'
+require 'socket'
+require 'condition_coverage/collector'
+
+if ARGV.first == '-h'
+  puts <<~HELP
+    Usage: condition_coverage_collector PORT DIR
+
+    Listens on PORT and saves each JSON report the :post reporter
+    sends as DIR/run-0001.json, DIR/run-0002.json, and so on,
+    creating DIR if need be. Runs until stopped.
+
+    Options:
+      -h    Show this help
+
+    Example:
+      condition_coverage_collector 4570 /tmp/condition/runs
+  HELP
+  exit 0
+end
+
+port, dir = ARGV
+FileUtils.mkdir_p(dir)
+server = TCPServer.new('0.0.0.0', Integer(port))
+ConditionCoverage::Collector.serve(server, dir)

data/lib/condition_coverage/auto.rb

@@ -0,0 +1,5 @@
+require_relative 'kernel'
+
+# Required by RUBYOPT=-rcondition_coverage/auto, to cover a run
+# with no change to its code, its settings all in ENV.
+ConditionCoverage.start_from_env(ENV)

data/lib/condition_coverage/collector.rb

@@ -0,0 +1,46 @@
+module ConditionCoverage
+  # Receives the JSON reports the :post reporter sends, so the
+  # runs that sent them, eg failing docker build steps, can
+  # be merged and rendered once they are all done.
+  module Collector
+    # Answers every request on server, forever, saving each
+    # body in dir as run-0001.json, run-0002.json, and so on,
+    # in the order they arrive. A connection sending no
+    # request, eg a check that the collector is up, is skipped.
+    def self.serve(server, dir)
+      n = 0
+      loop do
+        body = request_body(server.accept)
+        next if body.nil?
+
+        n += 1
+        File.write(format('%<dir>s/run-%<n>04d.json', dir:, n:), body)
+      end
+    end
+
+    # The body of the one HTTP request on client, after
+    # answering it 200; nil, unanswered, when client closes
+    # without sending a request.
+    def self.request_body(client)
+      headers = []
+      while (line = client.gets) && line != "\r\n"
+        headers << line
+      end
+      body = headers.empty? ? nil : answer(client, headers)
+      client.close
+      body
+    end
+
+    # The body of a request with headers, read from client,
+    # which is then answered 200.
+    def self.answer(client, headers)
+      length = headers.sum do |line|
+        name, value = line.split(':', 2)
+        name.casecmp?('content-length') ? value.to_i : 0
+      end
+      body = client.read(length)
+      client.write("HTTP/1.1 200 OK\r\nContent-Length: 0\r\n\r\n")
+      body
+    end
+  end
+end

data/lib/condition_coverage/configuration.rb

@@ -5,8 +5,8 @@ module ConditionCoverage
   # The settings given in a ConditionCoverage.start block.
   class Configuration
     GLOB_FLAGS = File::FNM_PATHNAME | File::FNM_EXTGLOB
-    SETTINGS = %w[cover coverage_dir report root].freeze
-    REPORTERS = %w[gaps html].freeze
+    SETTINGS = %w[collector cover coverage_dir report root].freeze
+    REPORTERS = %w[gaps html post].freeze
 
     # Evaluates the settings block against a new configuration.
     def initialize(&)
@@ -23,6 +23,14 @@ module ConditionCoverage
       @coverage_dir
     end
 
+    # The url the :post reporter sends the JSON report to.
+    attr_reader :collector_url
+
+    # Sets the url the :post reporter sends the JSON report to.
+    def collector(url)
+      @collector_url = url
+    end
+
     # Sets the dir that cover globs and site names are
     # relative to.
     def root(dir)

data/lib/condition_coverage/gaps.rb

@@ -1,13 +1,13 @@
 module ConditionCoverage
   # The terminal worklist: one line per missing outcome, and
   # nothing for a site seen both ways. Every count stays in
-  # the Result for reporters that show them all.
+  # the operands for reporters that show them all.
   module Gaps
-    # The gap lines of result, eg "k.rb:2:2 operand 0 never
-    # false", in manifest order.
-    def self.lines(result)
-      result.operands.flat_map do |site, operands|
-        site_gaps(site, operands)
+    # The gap lines of operands, as Result#operands gives them,
+    # eg "k.rb:2:2 operand 0 never false", in manifest order.
+    def self.lines(operands)
+      operands.flat_map do |site, counts|
+        site_gaps(site, counts)
       end
     end
 

data/lib/condition_coverage/html_report.rb

@@ -10,23 +10,25 @@ module ConditionCoverage
   # each covered file, written to the configuration's
   # output_dir.
   module HtmlReport
-    # Writes every page. files is { full path => {source:, sites:} }
-    # for each covered file, as the kernel parsed it.
-    def self.write(result, files, config)
+    # Writes every page. operands are as Result#operands gives
+    # them; files is { full path => {source:, sites:} } for each
+    # covered file, as the kernel parsed it.
+    def self.write(operands, files, config)
       dir = config.output_dir
-      index = HtmlIndex.page(Summary.files(result))
+      index = HtmlIndex.page(Summary.files(operands))
       File.write("#{dir}/index.html", index)
       files.each do |full_path, parsed|
         path = config.relative(full_path)
-        write_page("#{dir}/files/#{path}.html", path, parsed, result)
+        out = "#{dir}/files/#{path}.html"
+        write_page(out, path, parsed, operands)
       end
     end
 
     # Writes one file's page to out.
-    def self.write_page(out, path, parsed, result)
+    def self.write_page(out, path, parsed, operands)
       source = parsed[:source]
-      lines = SourceLines.of(source, parsed[:sites], result.operands)
-      trees = trees(source, parsed[:sites], lines, result.operands)
+      lines = SourceLines.of(source, parsed[:sites], operands)
+      trees = trees(source, parsed[:sites], lines, operands)
       FileUtils.mkdir_p(File.dirname(out))
       File.write(out, HtmlFile.page(path, lines, trees))
     end

data/lib/condition_coverage/json_writer.rb

@@ -4,10 +4,15 @@ module ConditionCoverage
   # The base reporter: the result as raw JSON, which every
   # other reporter can read back.
   module JsonWriter
-    # Writes result to path as {"sites": {site: [counts, ...]}}.
-    def self.write(result, path)
-      json = JSON.pretty_generate('sites' => result.operands)
-      File.write(path, json)
+    # Writes operands, as Result#operands gives them, to path as
+    # {"sites": {site: [counts, ...]}}.
+    def self.write(operands, path)
+      File.write(path, json(operands))
+    end
+
+    # operands as the JSON report's text.
+    def self.json(operands)
+      JSON.pretty_generate('sites' => operands)
     end
   end
 end

data/lib/condition_coverage/kernel.rb

@@ -1,10 +1,9 @@
 require 'fileutils'
 require_relative 'configuration'
-require_relative 'gaps'
-require_relative 'html_report'
 require_relative 'json_writer'
 require_relative 'manifest'
 require_relative 'recorder'
+require_relative 'render'
 require_relative 'result'
 require_relative 'rewriter'
 
@@ -47,6 +46,27 @@ module ConditionCoverage
     on_exit.call { report(config) }
   end
 
+  # Starts from settings in env alone, for a run whose code
+  # cannot be changed, eg one started by RUBYOPT=-r: covers
+  # the comma-separated CONDITION_COVERAGE_COVER globs under
+  # CONDITION_COVERAGE_ROOT, writing the JSON report into
+  # CONDITION_COVERAGE_DIR and POSTing it to
+  # CONDITION_COVERAGE_COLLECTOR. Without a collector it
+  # starts nothing, so RUBYOPT can be set for every run.
+  def self.start_from_env(env, on_exit: method(:at_exit))
+    return unless env.key?('CONDITION_COVERAGE_COLLECTOR')
+
+    start(on_exit:) do
+      root env.fetch('CONDITION_COVERAGE_ROOT')
+      env.fetch('CONDITION_COVERAGE_COVER').split(',').each do |glob|
+        cover glob
+      end
+      coverage_dir env.fetch('CONDITION_COVERAGE_DIR')
+      collector env.fetch('CONDITION_COVERAGE_COLLECTOR')
+      report :post
+    end
+  end
+
   # Routes every later load through instrumented, under
   # whichever configuration is current. Installed once: a
   # second definition would warn that it was redefined.
@@ -62,37 +82,28 @@ module ConditionCoverage
   def self.instrumented(path, config)
     return unless config.covers?(path)
 
-    parsed = @parsed[path] = parse(path, config)
+    parsed = @parsed[path] = Manifest.parse(path, config)
     rewritten = Rewriter.rewrite(parsed[:source], parsed[:sites])
     RubyVM::InstructionSequence.compile(rewritten, path, path)
   end
 
-  # The source and sites of the covered file at path.
-  def self.parse(path, config)
-    source = File.read(path)
-    { source:, sites: Manifest.sites(source, config.relative(path)) }
-  end
-
   # Writes the JSON report, then runs the chosen reporters, for
   # every covered file whether or not the run loaded it, into
   # coverage_dir, created first if need be.
   def self.report(config)
     files = parsed_files(config)
-    result = Result.new(merged_sites(files), @recorder.counts)
-    FileUtils.mkdir_p(config.output_dir)
-    JsonWriter.write(result, config.json_path)
-    run_reporters(result, files, config)
+    operands = collect(files, config)
+    Render.run(operands, files, config)
   end
 
-  # Runs each reporter chosen in the start block.
-  def self.run_reporters(result, files, config)
-    reporters = config.reporters
-    if reporters.include?(:gaps)
-      Gaps.lines(result).each { |line| puts line }
-    end
-    return unless reporters.include?(:html)
-
-    HtmlReport.write(result, files, config)
+  # The operands of every covered file, with the counts this
+  # run recorded, once written as the JSON report.
+  def self.collect(files, config)
+    sites = merged_sites(files)
+    operands = Result.new(sites, @recorder.counts).operands
+    FileUtils.mkdir_p(config.output_dir)
+    JsonWriter.write(operands, config.json_path)
+    operands
   end
 
   # { full path => {source:, sites:} } for every covered file:
@@ -100,7 +111,7 @@ module ConditionCoverage
   # never loaded.
   def self.parsed_files(config)
     config.files.to_h do |path|
-      [path, @parsed.fetch(path) { parse(path, config) }]
+      [path, @parsed.fetch(path) { Manifest.parse(path, config) }]
     end
   end
 

data/lib/condition_coverage/manifest.rb

@@ -13,6 +13,12 @@ module ConditionCoverage
       collector.sites
     end
 
+    # The source and sites of the covered file at full_path.
+    def self.parse(full_path, config)
+      source = File.read(full_path)
+      { source:, sites: sites(source, config.relative(full_path)) }
+    end
+
     # Collects each site and its Condition.
     class ConditionSites < Prism::Visitor
       # Starts with no sites found.
@@ -97,13 +103,22 @@ module ConditionCoverage
       ].freeze
       FALSY_TYPES = %i[nil_node false_node].freeze
 
+      # Ruby raises TypeError on assigning nil or false to these.
+      TRUTHY_GLOBALS = %i[$stdout $stderr].freeze
+
       # The one outcome a leaf can have, eg :truthy for `Clock.new`
       # or a literal, :falsy for `nil`; nil for a leaf that can
       # have either.
       def fixed_outcome(node)
         return :falsy if FALSY_TYPES.include?(node.type)
 
-        :truthy if TRUTHY_TYPES.include?(node.type) || new_call?(node)
+        :truthy if truthy_only?(node)
+      end
+
+      # True for a leaf that can never be falsy.
+      def truthy_only?(node)
+        TRUTHY_TYPES.include?(node.type) || new_call?(node) ||
+          truthy_global?(node)
       end
 
       # True for a call to new, eg `Clock.new`.
@@ -111,6 +126,12 @@ module ConditionCoverage
         node.type == :call_node && node.name == :new
       end
 
+      # True for $stdout or $stderr, which can never be falsy.
+      def truthy_global?(node)
+        node.type == :global_variable_read_node &&
+          TRUTHY_GLOBALS.include?(node.name)
+      end
+
       # True for a call to a named method with arguments and no
       # parentheses, eg `id.include? 'N'`, but not `id.size == 6`.
       def command?(node)

data/lib/condition_coverage/merge.rb

@@ -0,0 +1,22 @@
+module ConditionCoverage
+  # Combines the operands of several runs of the same source.
+  module Merge
+    # One operands hash, as Result#operands gives them, whose
+    # counts are the sums of every run's.
+    def self.operands(runs)
+      runs.reduce do |total, run|
+        total.merge(run) do |_site, mine, theirs|
+          mine.zip(theirs).map { |a, b| sum(a, b) }
+        end
+      end
+    end
+
+    # One operand's counts from two runs, added; a fixed leaf's
+    # only comes from the source, so either run's will do.
+    def self.sum(mine, theirs)
+      mine.merge(theirs) do |key, a, b|
+        key == :only ? a : a + b
+      end
+    end
+  end
+end

data/lib/condition_coverage/post.rb

@@ -0,0 +1,24 @@
+require 'net/http'
+require_relative 'json_writer'
+
+module ConditionCoverage
+  # The :post reporter: sends the JSON report to a collector,
+  # for runs whose own files are thrown away, eg a failing
+  # docker build step.
+  module Post
+    # POSTs operands, as the JSON report, to url, never through
+    # an http_proxy, which could not reach a collector local to
+    # the run. A collector that cannot be reached, or does not
+    # answer within a second, is passed over in silence: the
+    # run's own output and exit status must stay as they are.
+    def self.deliver(operands, url)
+      uri = URI(url)
+      http = Net::HTTP.new(uri.host, uri.port, nil)
+      http.open_timeout = http.read_timeout = 1
+      headers = { 'Content-Type' => 'application/json' }
+      http.post(uri.path, JsonWriter.json(operands), headers)
+    rescue StandardError
+      nil
+    end
+  end
+end

data/lib/condition_coverage/render.rb

@@ -0,0 +1,55 @@
+require 'json'
+require_relative 'gaps'
+require_relative 'html_report'
+require_relative 'manifest'
+require_relative 'merge'
+require_relative 'post'
+
+module ConditionCoverage
+  # The reporters, run from operands alone: needing nothing the
+  # run recorded beyond them, they can run in another process.
+  module Render
+    # Runs each reporter chosen in config. operands are as
+    # Result#operands gives them; files is { full path =>
+    # {source:, sites:} } for each covered file.
+    def self.run(operands, files, config)
+      reporters = config.reporters
+      if reporters.include?(:gaps)
+        Gaps.lines(operands).each { |line| puts line }
+      end
+      if reporters.include?(:html)
+        HtmlReport.write(operands, files, config)
+      end
+      return unless reporters.include?(:post)
+
+      Post.deliver(operands, config.collector_url)
+    end
+
+    # Runs the reporters chosen in config from the JSON reports
+    # at paths, their counts merged, parsing each covered file
+    # afresh. paths is config.json_path alone by default.
+    def self.from_json(config, paths = [config.json_path])
+      operands = Merge.operands(paths.map { |path| read(path) })
+      files = config.files.to_h do |path|
+        [path, Manifest.parse(path, config)]
+      end
+      run(operands, files, config)
+    end
+
+    # The operands of the JSON report at path.
+    def self.read(path)
+      json = JSON.parse(File.read(path))
+      json['sites'].transform_values do |leaves|
+        leaves.map { |leaf| counts(leaf) }
+      end
+    end
+
+    # One operand's counts as JSON holds them, with the keys,
+    # and any only: value, as the symbols Result#operands uses.
+    def self.counts(leaf)
+      counts = { truthy: leaf['truthy'], falsy: leaf['falsy'] }
+      only = leaf['only']
+      only ? counts.merge(only: only.to_sym) : counts
+    end
+  end
+end

data/lib/condition_coverage/summary.rb

@@ -1,9 +1,10 @@
 module ConditionCoverage
   # The per-file numbers the HTML index page shows.
   module Summary
-    # One row per file that has sites in result, in path order.
-    def self.files(result)
-      by_path = result.operands.group_by { |site, _| path_of(site) }
+    # One row per file that has sites in operands, as
+    # Result#operands gives them, in path order.
+    def self.files(operands)
+      by_path = operands.group_by { |site, _| path_of(site) }
       by_path.sort.map { |path, sites| row(path, sites) }
     end
 

data/lib/condition_coverage/version.rb

@@ -1,3 +1,3 @@
 module ConditionCoverage
-  VERSION = '0.0.5'.freeze
+  VERSION = '0.0.6'.freeze
 end

metadata

@@ -1,11 +1,11 @@
 --- !ruby/object:Gem::Specification
 name: condition_coverage
 version: !ruby/object:Gem::Version
-  version: 0.0.5
+  version: 0.0.6
 platform: ruby
 authors:
 - Jon Jagger
-bindir: bin
+bindir: exe
 cert_chain: []
 date: 1980-01-02 00:00:00.000000000 Z
 dependencies:
@@ -25,13 +25,17 @@ dependencies:
         version: '1.0'
 description: Records whether each operand of every &&, ||, and and or condition was
   seen both true and false, and reports the gaps in the terminal and as HTML.
-executables: []
+executables:
+- condition_coverage_collector
 extensions: []
 extra_rdoc_files: []
 files:
 - LICENSE.md
 - README.md
+- exe/condition_coverage_collector
 - lib/condition_coverage.rb
+- lib/condition_coverage/auto.rb
+- lib/condition_coverage/collector.rb
 - lib/condition_coverage/condition.rb
 - lib/condition_coverage/configuration.rb
 - lib/condition_coverage/gaps.rb
@@ -49,7 +53,10 @@ files:
 - lib/condition_coverage/json_writer.rb
 - lib/condition_coverage/kernel.rb
 - lib/condition_coverage/manifest.rb
+- lib/condition_coverage/merge.rb
+- lib/condition_coverage/post.rb
 - lib/condition_coverage/recorder.rb
+- lib/condition_coverage/render.rb
 - lib/condition_coverage/result.rb
 - lib/condition_coverage/rewriter.rb
 - lib/condition_coverage/site.rb