condition_coverage 0.0.1 → 0.0.3

checksums.yaml

@@ -1,7 +1,7 @@
 ---
 SHA256:
-  metadata.gz: 8fbd7cd27ab04963d5c91539a6accc52d6f178eef85858b48e6b553dd51f39de
-  data.tar.gz: 59cebe77670d20308973d24f96712368a204adfb32ac90a7e0497f78aabf9331
+  metadata.gz: e0b1aba7a743838c35d3b4d139b40efebafd48edef44b82ffc48f7c045eb6861
+  data.tar.gz: '074496c6beaf83b9c6ec438019eb20e7b4f8231230ba94cd5e7a87cc2dc01eb1'
 SHA512:
-  metadata.gz: ff8f96409d90bb54be22e3755e461849c2f9dfaf5a0ca0fbf5bb685b64c2066a22385e5cf7bfec2bcebd12ddf8687555d56112e107f5c242a33e7a5156977ce5
-  data.tar.gz: 86324f587f50156ff2ae9049f7d019fd1eb17b984f4ac0b45a2e0ecbccd114b1d849d9e8cdde4dd824d7e5e047e153f7e9d622474f811e9bcaeb471b38159179
+  metadata.gz: debb313edef170467b8146405bc25f212b852798c19492b20699a111d255c05015ade7357a90cc9666e9a5642674fa31a0dcea3f930978bd2ef8bad2b37cab04
+  data.tar.gz: e90aa8e71930f57e5e50b5f70155d7166ace8741f300abb7092bafbb3d47bafaaa3ce409cfe7102fc1795dfe7cea5224961a8497e027a617351b371010d83432

data/README.md

@@ -1,75 +1,152 @@
 # condition_coverage
 
-Status: an outline of the idea. No working code is published yet.
-
 Condition coverage for Ruby, to run alongside SimpleCov's line and branch
 coverage.
 
+Status: 0.0.3 covers `&&`, `||`, `and` and `or`: each leaf operand of a
+condition, however its operators are chained, nested or grouped. Decisions
+hidden inside calls come next (see Roadmap).
+
+Demo: https://cyber-dojo.github.io/condition_coverage/ shows the HTML
+report for the mock source tree in `demo/`. `make demo` builds and opens
+the same report locally.
+
 ## The gap it fills
 
 Ruby's built-in `Coverage` (which SimpleCov reads) tracks branches, not the
 conditions inside them:
 
 ```ruby
-if a || b || c
+return :ok if ok?(id) && id.start_with?('N')
 ```
 
-is two branches. Tests that reach both can still leave `b` never true and `c`
-never evaluated, and branch coverage reports 100%.
-
-## How it works
+is two branches. Tests that reach both can still leave `id.start_with?('N')`
+never false, or never evaluated, and branch coverage reports 100%.
 
-At load time, each file under the covered root is parsed with Prism and each
-`&&` / `||` operand is wrapped in a recorder call, before compilation:
+## Use
 
 ```ruby
-if __cc(7, 0, a) || __cc(7, 1, b) || __cc(7, 2, c)
+# test coverage config, before any application code is required
+require 'simplecov'
+require 'condition_coverage'
+SimpleCov.start do
+  # ...
+end
+ConditionCoverage.start do
+  root APP_DIR                 # cover globs and site names are relative to it
+  cover 'source/**/*.rb'       # each matching file is instrumented as it loads
+  coverage_dir '/tmp/coverage' # where condition_coverage.json is written
+  report :gaps, :html          # the reporters run at exit (default :gaps)
+end
 ```
 
-The recorder notes whether the operand was truthy or falsy and returns it
-unchanged, so short-circuiting and the value of the expression are preserved.
-The rewrite happens in memory only; no file on disk changes.
+A misspelt setting or an unknown reporter is refused by name, with the list
+of the ones there are. Calling `start` again replaces the configuration.
+
+## What a run reports
+
+As the process exits:
+
+- `condition_coverage.json` is always written to `coverage_dir`: every site
+  of every covered file, with each operand's true and false counts. It is
+  the base the other reporters read.
+- `:gaps` prints one line per missing outcome, in manifest order:
 
-A static manifest of every site (file, line, column, operand count) is built
-by scanning the covered files, so an operand that never ran, or a file never
-loaded, is reported as missed rather than silently absent.
+```text
+source/kata.rb:3:23 operand 1 never false
+source/kata.rb:4:24 never reached
+```
+
+A site is named by the `path:line:column` where its condition starts, the
+path relative to `root`. A site in a covered file the run never loaded is
+still reported, as `never reached`.
 
-## Beyond and/or
+A site is an outermost condition. Its operands are numbered left to right
+across the whole of it, nested conditions included:
 
-The same hook can record decisions hidden inside calls, each with its own
-outcome classes:
+```ruby
+ok?(id) && new?(id) && id.size == 6     # one && site, operands 0 1 2
+ok?(id) && (new?(id) || id.size == 6)   # an || nested in an && site, 0 1 2
+ok?(id) && new?(id) || id.size == 6     # an && nested in an || site, 0 1 2
+ids.any? { |k| k && k.ok? } && ok?(id)  # two sites: the block's is its own
+```
 
-- `any?` / `all?` / `none?`: empty, decided by the first item, decided later,
-  never decided
-- `find` / `detect`: found vs nil
-- `fetch(k, default)`, `dig`, `Hash#[]` with a default: present vs default
-- `x || default` in value context, `&.` safe navigation: nil vs not nil
-- ternaries, `unless`, modifier `if`, `case`/`when` and `in` patterns
-- regex matches: nil vs MatchData
+A chain of one operator is flat; a change of operator, by precedence or by
+parentheses, nests. Parentheses are not part of the tree, since they decide
+nothing.
 
-## Reporting
+- `:html` writes `index.html` to `coverage_dir`: the covered files with
+  their sites, outcomes seen (with a percent) and outcomes missing, each
+  column sortable. Each file name opens `files/<path>.html`, the file's
+  source. On a line holding a condition the line number is a button (blue
+  and outlined when every outcome was seen, solid orange when any is
+  missing) that opens a tree of the condition: its operators, nested as
+  Ruby groups them, and each leaf operand's true and false counts and its
+  gap. A condition written over several lines has one button over all of
+  its line numbers.
 
-Gaps only, in the style of a Missing column. Each site shows its source line
-with a caret under each operand that was never true, never false, or never
-evaluated, plus a JSON form of the same data. An HTML view underlines each
-operand in its own colour.
+## How it works
 
-## Intended use
+At load time each covered file is read and parsed once, with Prism. The
+parse gives the manifest: each site's condition tree and the byte range of
+each of its leaf operands. The rewriter then inserts a record call around
+each leaf, without parsing again, and the result is compiled in place of
+the file:
 
 ```ruby
-# test coverage config, before any application code is required
-require 'simplecov'
-require 'condition_coverage'
-SimpleCov.start do
-  # ...
-end
-ConditionCoverage.start(root: APP_DIR, cover: 'source/**/*.rb')
+ok?(id) && new?(id)
+# compiles as
+ConditionCoverage.record("k.rb:2:2", 0, ok?(id)) && ConditionCoverage.record("k.rb:2:2", 1, new?(id))
 ```
 
-The figure is gathered and reported, not gated.
+A command call such as `ok? id` (after `and`) is not valid as a bare
+argument, so that operand alone gets parentheses of its own. No other operand
+does: parentheses make a statement, and a statement on a continuation line
+gives that line a line-coverage count of its own, which would change
+SimpleCov's line totals. With that one exception, an instrumented file has
+the same relevant lines as an uninstrumented one.
+
+`record` counts whether the operand was truthy or falsy and returns it
+unchanged, so short-circuiting and the value of the expression are
+preserved. Only text is inserted, never a newline, so backtrace line numbers
+still match the file. The rewrite happens in memory; no file on disk
+changes. A condition with a single operand is left alone, since branch
+coverage already sees both its outcomes.
+
+At exit the manifests parsed during the run are reused; only covered files
+the run never loaded are read and parsed then.
+
+## Roadmap
+
+- decisions hidden inside calls, each with its own outcome classes:
+  - `any?` / `all?` / `none?`: empty, decided by the first item, decided
+    later, never decided
+  - `find` / `detect`: found vs nil
+  - `fetch(k, default)`, `dig`, `Hash#[]` with a default: present vs default
+  - `x || default` in value context, `&.` safe navigation: nil vs not nil
+  - ternaries, `unless`, modifier `if`, `case`/`when` and `in` patterns
+  - regex matches: nil vs MatchData
 
 ## Constraints
 
 - CRuby only: the hook is `RubyVM::InstructionSequence.load_iseq`.
+- Ruby 3.2 or later, for `String#bytesplice`.
 - That hook has a single slot, which Bootsnap also uses, so the two cannot
   both be active in one process.
+
+## Development
+
+`make all` runs, in a Docker image built from the repo's `Dockerfile`:
+
+- `make lint`: rubocop over `lib/`, `test/` and `bin/` alike
+- `make test`: the tests, then a check that the coverage report has zero
+  missed lines and branches in both the code and the tests, within the
+  bounds in `test/coverage_metrics_params.json`
+  (`make test tids='6D1K'` runs only matching tests, and skips the check)
+- `make test_check`: shunit2 tests showing that a breached bound fails the
+  check with a non-zero exit
+
+`make demo` (not part of `make all`) runs `demo/run.rb` through
+`bin/build_demo.sh`, writing the report to `site/` and opening it. The
+Pages workflow (`.github/workflows/pages.yml`) runs the same script on
+each push to `main` and publishes `site/`.

data/lib/condition_coverage/condition.rb

@@ -0,0 +1,20 @@
+module ConditionCoverage
+  # One site's condition as the manifest found it: its operator
+  # and its operands, each the byte range of a leaf or a nested
+  # Condition. A site's commands are the byte ranges of its
+  # leaves that are command calls, eg `ok? id`.
+  Condition = Data.define(:operator, :operands, :commands) do
+    # A condition with no command-call leaves by default.
+    def initialize(operator:, operands:, commands: [])
+      super
+    end
+
+    # The byte range of every leaf operand, left to right; a
+    # leaf's index here is the index it is recorded under.
+    def leaves
+      operands.flat_map do |operand|
+        operand.is_a?(Condition) ? operand.leaves : [operand]
+      end
+    end
+  end
+end

data/lib/condition_coverage/configuration.rb

@@ -0,0 +1,98 @@
+module ConditionCoverage
+  # Raised for a start block that cannot be applied.
+  class ConfigurationError < StandardError; end
+
+  # The settings given in a ConditionCoverage.start block.
+  class Configuration
+    GLOB_FLAGS = File::FNM_PATHNAME | File::FNM_EXTGLOB
+    SETTINGS = %w[cover coverage_dir report root].freeze
+    REPORTERS = %w[gaps html].freeze
+
+    # Evaluates the settings block against a new configuration.
+    def initialize(&)
+      @globs = []
+      @reporters = [:gaps]
+      instance_eval(&)
+    end
+
+    # The reporters chosen with report, :gaps when none were.
+    attr_reader :reporters
+
+    # The dir the reports are written to.
+    def output_dir
+      @coverage_dir
+    end
+
+    # Sets the dir that cover globs and site names are
+    # relative to.
+    def root(dir)
+      @root = dir
+    end
+
+    # Adds a glob, relative to root, of files to instrument.
+    def cover(glob)
+      @globs << glob
+    end
+
+    # Chooses the reporters run at exit, eg report :gaps;
+    # refuses a name there is no reporter for.
+    def report(*names)
+      names.each do |name|
+        next if REPORTERS.include?(name.to_s)
+
+        raise ConfigurationError, [
+          "unknown reporter #{name} in ConditionCoverage.start",
+          "the reporters are: #{REPORTERS.join(', ')}"
+        ].join("\n")
+      end
+      @reporters = names.map(&:to_sym)
+    end
+
+    # Sets the dir the condition_coverage.json report is
+    # written to.
+    def coverage_dir(dir)
+      @coverage_dir = dir
+    end
+
+    # Every file the cover globs match under root, as full
+    # paths, whether or not the run loads it.
+    def files
+      Dir.glob(@globs, base: @root).sort.map { |f| "#{@root}/#{f}" }
+    end
+
+    # Where the JSON report goes.
+    def json_path
+      "#{output_dir}/condition_coverage.json"
+    end
+
+    # True when path is under root and matches a cover glob.
+    def covers?(path)
+      path.start_with?("#{@root}/") && matches?(relative(path))
+    end
+
+    # True when the root-relative path matches a cover glob.
+    def matches?(relative_path)
+      @globs.any? do |glob|
+        File.fnmatch?(glob, relative_path, GLOB_FLAGS)
+      end
+    end
+
+    # path relative to root, eg "source/kata.rb".
+    def relative(path)
+      path.delete_prefix("#{@root}/")
+    end
+
+    # Refuses a setting that does not exist, naming it.
+    def method_missing(name, *)
+      raise ConfigurationError, [
+        "unknown setting #{name} in ConditionCoverage.start",
+        "the settings are: #{SETTINGS.join(', ')}"
+      ].join("\n")
+    end
+
+    # Answers only for the settings there are.
+    def respond_to_missing?(_name, _include_private = false)
+      false
+    end
+  end
+end

data/lib/condition_coverage/gaps.rb

@@ -0,0 +1,42 @@
+module ConditionCoverage
+  # The terminal worklist: one line per missing outcome, and
+  # nothing for a site seen both ways. Every count stays in
+  # the Result for reporters that show them all.
+  module Gaps
+    # The gap lines of result, eg "k.rb:2:2 operand 0 never
+    # false", in manifest order.
+    def self.lines(result)
+      result.operands.flat_map do |site, operands|
+        site_gaps(site, operands)
+      end
+    end
+
+    # One line for a site that never ran, else one per gap
+    # in its operands.
+    def self.site_gaps(site, operands)
+      if operands.all? { |counts| never_ran?(counts) }
+        return ["#{site} never reached"]
+      end
+
+      operands.each_with_index.flat_map do |counts, index|
+        operand_gap(counts).map do |gap|
+          "#{site} operand #{index} #{gap}"
+        end
+      end
+    end
+
+    # The one outcome this operand is missing, if any.
+    def self.operand_gap(counts)
+      return ['never evaluated'] if never_ran?(counts)
+      return ['never true'] if counts[:truthy].zero?
+      return ['never false'] if counts[:falsy].zero?
+
+      []
+    end
+
+    # True when the operand was never recorded either way.
+    def self.never_ran?(counts)
+      counts[:truthy].zero? && counts[:falsy].zero?
+    end
+  end
+end

data/lib/condition_coverage/html/file.html

@@ -0,0 +1,27 @@
+<!DOCTYPE html>
+<html lang="en">
+<head>
+<meta charset="utf-8"/>
+<meta name="viewport" content="width=device-width, initial-scale=1"/>
+<title>{{path}} · Condition coverage</title>
+<style>
+/*<![CDATA[*/
+{{style}}
+/*]]>*/
+</style>
+</head>
+<body>
+<main class="page">
+<p class="back"><a data-index="" href="{{index}}">Condition coverage</a></p>
+<h1 class="path">{{path}}</h1>
+<div class="box">
+{{listing}}
+</div>
+</main>
+<script>
+//<![CDATA[
+{{script}}
+//]]>
+</script>
+</body>
+</html>

data/lib/condition_coverage/html/index.html

@@ -0,0 +1,38 @@
+<!DOCTYPE html>
+<html lang="en">
+<head>
+<meta charset="utf-8"/>
+<meta name="viewport" content="width=device-width, initial-scale=1"/>
+<title>Condition coverage</title>
+<style>
+/*<![CDATA[*/
+{{style}}
+/*]]>*/
+</style>
+</head>
+<body>
+<main class="page">
+<h1>Condition coverage</h1>
+<div class="box">
+<table id="files">
+<thead>
+<tr>
+<th aria-sort="none"><button type="button" data-column="0" data-type="text">File</button></th>
+<th class="num" aria-sort="none"><button type="button" data-column="1" data-type="number">Sites</button></th>
+<th class="num" aria-sort="none"><button type="button" data-column="2" data-type="number">Outcomes seen</button></th>
+<th class="num" aria-sort="none"><button type="button" data-column="3" data-type="number">Missing</button></th>
+</tr>
+</thead>
+<tbody>
+{{rows}}
+</tbody>
+</table>
+</div>
+</main>
+<script>
+//<![CDATA[
+{{script}}
+//]]>
+</script>
+</body>
+</html>

data/lib/condition_coverage/html/sort.js

@@ -0,0 +1,28 @@
+// Sorts the files table by the clicked column's data-sort values;
+// a second click on the same column reverses the order.
+(function () {
+  var table = document.getElementById('files');
+  var body = table.tBodies[0];
+  var state = { column: -1, ascending: true };
+
+  table.querySelectorAll('thead button').forEach(function (button) {
+    button.addEventListener('click', function () {
+      var column = Number(button.dataset.column);
+      var numeric = button.dataset.type === 'number';
+      state.ascending = state.column === column ? !state.ascending : true;
+      state.column = column;
+      var rows = Array.from(body.rows);
+      rows.sort(function (a, b) {
+        var x = a.cells[column].dataset.sort;
+        var y = b.cells[column].dataset.sort;
+        var order = numeric ? Number(x) - Number(y) : x.localeCompare(y);
+        return state.ascending ? order : -order;
+      });
+      rows.forEach(function (row) { body.appendChild(row); });
+      table.querySelectorAll('thead th').forEach(function (th, i) {
+        var sort = state.ascending ? 'ascending' : 'descending';
+        th.setAttribute('aria-sort', i === column ? sort : 'none');
+      });
+    });
+  });
+})();

data/lib/condition_coverage/html/style.css

@@ -0,0 +1,106 @@
+/* Blue (seen both ways) and orange (missing) stay distinct for
+   red-green colour blindness; states also differ in form. */
+:root {
+  --bg: #FBFBFC;
+  --surface: #FFFFFF;
+  --panel: #F2F4F7;
+  --fg: #1C2230;
+  --muted: #5B6577;
+  --rule: #D5DAE2;
+  --ok: #1F5FBF;
+  --ok-tint: #E6EEFA;
+  --warn: #A35200;
+  --bad: #B54708;
+  --on-status: #FFFFFF;
+  --link: #1F5FBF;
+  --font-body: "Helvetica Neue", Arial, sans-serif;
+  --font-code: Menlo, Consolas, "DejaVu Sans Mono", monospace;
+}
+@media (prefers-color-scheme: dark) {
+  :root {
+    --bg: #12151C; --surface: #181C25; --panel: #212633; --fg: #E4E8EF;
+    --muted: #9AA4B5; --rule: #323949; --ok: #7FB0FF; --ok-tint: #1A2740;
+    --warn: #F0A35E; --bad: #F08A3C; --on-status: #12151C; --link: #7FB0FF;
+    color-scheme: dark;
+  }
+}
+body { margin: 0; background: var(--bg); color: var(--fg); font-family: var(--font-body); font-size: 15px; }
+a { color: var(--link); }
+.page {
+  max-width: 1080px; margin: 0 auto; display: flex; flex-direction: column; gap: 24px;
+  padding: 40px max(16px, env(safe-area-inset-right, 0px)) 64px max(16px, env(safe-area-inset-left, 0px));
+}
+h1 { margin: 0; font-size: 28px; font-weight: 600; }
+.box { border: 1px solid var(--rule); border-radius: 8px; background: var(--surface); overflow-x: auto; }
+table { width: 100%; border-collapse: collapse; font-size: 14px; font-variant-numeric: tabular-nums; }
+th, td { padding: 10px 16px; text-align: left; }
+th { background: var(--panel); }
+td { border-top: 1px solid var(--rule); }
+td:first-child { font-family: var(--font-code); }
+.num, td:not(:first-child) { text-align: right; }
+th button {
+  font: 600 14px var(--font-body); color: var(--fg); background: none; border: 0;
+  padding: 4px 0; cursor: pointer;
+}
+th button::after { content: " \2195"; color: var(--muted); }
+th[aria-sort="ascending"] button::after { content: " \25B2"; color: var(--fg); }
+th[aria-sort="descending"] button::after { content: " \25BC"; color: var(--fg); }
+th button:focus-visible { outline: 3px solid var(--link); outline-offset: 2px; }
+
+/* A file's page: plain source; on lines holding a condition the
+   line numbers are one button. Every number has the same box, so
+   digits stay aligned whether or not they are on a button. */
+.back { margin: 0; font-size: 14px; }
+h1.path { font-family: var(--font-code); font-size: 20px; }
+.code { font-family: var(--font-code); font-size: 14px; line-height: 32px; padding: 10px 0; min-width: 720px; }
+.line, .block { display: flex; align-items: flex-start; }
+.srcs { display: flex; flex-direction: column; }
+.src { white-space: pre; }
+.lineno {
+  display: block; width: 52px; padding-right: 20px; text-align: right;
+  color: var(--muted); font-variant-numeric: tabular-nums; flex-shrink: 0;
+}
+button.lnum {
+  display: flex; flex-direction: column; flex-shrink: 0; margin: 4px 0; padding: 0;
+  border: 1px solid transparent; border-radius: 6px; cursor: pointer;
+  font: 600 14px/24px var(--font-code); box-shadow: 0 1px 3px rgba(0, 0, 0, 0.22);
+}
+button.lnum { position: relative; }
+button.lnum::before {
+  content: "\25B8"; position: absolute; left: 7px; top: 4px;
+  font-size: 11px; line-height: 24px; transition: transform 120ms;
+}
+button.lnum[aria-expanded="true"]::before { transform: rotate(90deg); }
+/* With a site count the count and the triangle share the
+   button's height, one 11px line each. */
+button.lnum:has(.site-count)::before { top: 12px; line-height: 11px; }
+/* On a button over two or more lines the triangle sits at the
+   very bottom. */
+button.lnum:has(.lineno + .lineno)::before { top: auto; bottom: 2px; line-height: 11px; }
+.site-count {
+  position: absolute; left: 7px; top: 1px;
+  font-size: 9px; line-height: 11px; font-weight: 700;
+}
+/* The rule between sites spans the tree's full width, through
+   its 16px side padding. */
+.site + .site {
+  border-top: 1px solid var(--rule); margin: 6px -16px 0; padding: 6px 16px 0;
+}
+button.lnum .lineno { width: 50px; padding-right: 19px; color: inherit; line-height: 32px; margin: -4px 0; }
+button.lnum[data-status="covered"] { background: var(--ok-tint); color: var(--ok); border-color: var(--ok); }
+button.lnum[data-status="gaps"] { background: var(--bad); color: var(--on-status); border-color: var(--bad); }
+button.lnum:hover { filter: brightness(1.1); }
+button.lnum:focus-visible { outline: 3px solid var(--link); outline-offset: 2px; }
+.tree {
+  margin: 4px 20px 12px 72px; padding: 10px 16px; border: 1px solid var(--rule);
+  border-radius: 8px; background: var(--panel); line-height: 30px;
+}
+.row { display: grid; grid-template-columns: auto minmax(200px, 1fr) 70px 70px 160px; column-gap: 16px; }
+.row.headings { font-family: var(--font-body); font-size: 12px; color: var(--muted); }
+.row.headings span:first-child { grid-column: span 2; }
+.row.headings span:nth-child(2), .row.headings span:nth-child(3) { text-align: right; }
+.operator { grid-column: span 2; font-weight: 600; }
+.branch { white-space: pre; color: var(--muted); margin-right: -16px; }
+.row .text { white-space: pre; }
+.row .truthy, .row .falsy { text-align: right; font-variant-numeric: tabular-nums; }
+.row .gap { color: var(--bad); }

data/lib/condition_coverage/html/toggle.js

@@ -0,0 +1,12 @@
+// Opens and closes the tree a line-number button names in its
+// aria-controls, keeping aria-expanded in step.
+(function () {
+  document.querySelectorAll('button[aria-controls]').forEach(function (button) {
+    button.addEventListener('click', function () {
+      var tree = document.getElementById(button.getAttribute('aria-controls'));
+      var open = button.getAttribute('aria-expanded') === 'true';
+      button.setAttribute('aria-expanded', open ? 'false' : 'true');
+      tree.hidden = open;
+    });
+  });
+})();