sloplint 0.8.1 → 0.9.0

checksums.yaml

@@ -1,7 +1,7 @@
 ---
 SHA256:
-  metadata.gz: 7b924df142be1939d6da4e27d2e925b2de378cbe835e5c9ea41e3c3b0b046633
-  data.tar.gz: 128a3a13dd8d4dc81610266ede34b52e73293b61af560ff7fa26d9dbd9708dd5
+  metadata.gz: 9031ddb2a2a80637f1958c3ec24014261a23b017c25cc3032e72e85f9acf0313
+  data.tar.gz: 53389164a5fdf74367d985bc1d3eff0d661c47cb70d22469ab89e55f56e4ac22
 SHA512:
-  metadata.gz: ffc7954ff2c5ccab7e431e007be51bddbb0a42cf27fe54a1c8629ec90c9c0c47c9a7ceba91bc03116f38303241efcf7b4263a84376aedae67a95dae2877d3db8
-  data.tar.gz: f0c893e80441a5e738a5a037897a064b9cf201fb1bca6f5ca5e1b160eabbf2d5c7d870800a0d8ac36263b00a8dbc0ca331673da4f302a3e2e4a3f41346248e3e
+  metadata.gz: 2029db5c55bdaed221ff62c629eaf9989bf05044274f340295ecb5122ba5cd063c773566c9c9bf8e201050348284103f6910aed13b6b24745b538ed030738612
+  data.tar.gz: 46130960f17c5fd70b7a8752065fc85520b8eeac94498588b3d0b798daa3bdb8683ab5084e258e561a9b502227793d9746c0fc7e2d7ab2f143a0b5f7364c2fc7

data/CHANGELOG.md

@@ -3,7 +3,110 @@
 All notable changes to this project are documented here. Format loosely
 follows [Keep a Changelog](https://keepachangelog.com/en/1.1.0/).
 
-## [Unreleased]
+## sloplint-judge
+
+### [0.1.0] - 2026-09-21
+
+- New sentence rule `unnamed-authority` (`info`, `medium`): a claim handed to
+  experts, studies, research, critics or many, an authority the reader could
+  not find and that speaks for nobody. Officials, a spokesperson and the
+  other sources news quotes by convention pass, and so does an abstract's
+  prior work. The reading behind the regex `vague-attribution`.
+- New sentence rule `stated-stakes` (`info`, `low`, off by default): a sentence
+  that says something is crucial, vital or key and gives no fact, number or
+  consequence, in it or in the sentence after it. Off by default because the
+  model rarely answers it above low confidence; `--select stated-stakes` or
+  `--strict` runs it.
+- A rule at `low` confidence (`same-weight`, `matched-shape`) now reports its
+  notes when `--select` names it; before, they ran and printed nothing
+  without `--strict`. The rule's `low` caps the note's confidence; what a
+  default run drops is an answer the model itself gave at low confidence.
+- `check --judge` always writes the `{"notes", "judge"}` object, with 0
+  requests when no judge rule survived selection. `compare` accepts
+  `--drift` after the two files and refuses a third. `status` refuses a key
+  with a control character the way `check --judge` does.
+- Graveyard: `owned-claim` was retried as `no-actor` and stays out, and
+  `buried-verbs`, the paragraph reading of the nominalization shift, joins
+  it; neither crossed the flag on any side. The entries in `docs/JUDGE.md`
+  say so.
+- New paragraph rule `promotional` (`warning`, `medium`): a paragraph in
+  which every judgment is favorable, none comes with a measure and no
+  drawback appears. The paragraph-level reading behind `puffery-words`,
+  which catches the register after the watch words have aged out.
+- New paragraph rule `self-narration` (`info`, `medium`): a paragraph whose
+  sentences signpost the document, what comes first, what a section covers,
+  what the reader should take away, instead of saying something about the
+  subject. `throat-clearing` sees the first sentence; this is the paragraph.
+- `script/calibrate run` also prints how many units each side would have
+  flagged at the confidence `check` reports. A rare flag barely moves the
+  rank statistic, so for such a rule the count is the number to quote.
+- New paragraph rule `same-weight` (`info`, `low`, off by default): a paragraph
+  that states its inferences and opinions as flatly as its measurements, with
+  no probably, no we think, and no reason given. Separates model from human
+  text in news and abstracts; off by default because a design document argues
+  in flat sentences on purpose. `--select same-weight` or `--strict` runs it.
+- New sentence rule `trailing-gloss` (`info`, `medium`): a sentence that ends
+  on a comma and an -ing clause that interprets the fact before it
+  ("highlighting the value of", "underscoring the importance of") rather than
+  adding a fact or a consequence. The reading the regex
+  `trailing-significance-participle` could not do.
+- Graveyard: `redundancy` was retried as `restatement` with the fairness
+  reading it was owed, reversed in news against every model side, and stays
+  out. The entry in `docs/JUDGE.md` says why.
+- `wrap-up` also flags a last sentence that admits problems and then
+  promises a bright future ("Despite these challenges, the future looks
+  promising"). Its flagged level says so, its message now reads "Paragraph
+  ends on a summary, a moral or a hope", and fixtures pin both sides: the
+  hope that names nothing is flagged, the same frame closing on a dated
+  fact is not. Wikipedia's field guide documents the formula across
+  unrelated topics, and the GPT family ends documents on it.
+
+- First release of the judge gem: the rule catalog in `docs/JUDGE.md`, the
+  Jev backend, `check`, `compare`, `rules`, `explain`, and `script/calibrate`.
+  Requires sloplint 0.9; see "Phase two" in `docs/JUDGE.md` for what is next.
+- `check` reports what the judge spent: JSON output is `{"notes", "judge"}`
+  with the backend, request count, token counts and cost in dollars under
+  `judge`, and the same line goes to stderr. Jev returns no price, so the
+  cost is computed from TypeSafe's public price: $42 per billion input
+  tokens, output tokens free.
+- The key can live in the OS keychain instead of the environment.
+  `sloplint-judge key set` stores it once (the keychain tool prompts, so the
+  key is never on a command line), the Jev adapter reads it after
+  `TYPESAFE_API_KEY`, and `sloplint-judge status` says whether a run could
+  happen and where the key is, without reading it. `sloplint-judge key
+  unset` removes the item. The check skill probes with `status`.
+- `SYSTEMONE_URL` must point at a `typesafe.ai` host, not only be `https`:
+  the key and the document go there, so an injected URL on the sanctioned
+  command is refused with exit 2.
+
+## sloplint
+
+## [0.9.0] - 2026-09-21
+
+### Added
+
+- `sloplint check --judge` runs the rules of sloplint-judge alongside the
+  regex catalog and merges the notes in document order. The judge is a second
+  gem built from this repository (`sloplint-judge.gemspec`, `exe/sloplint-judge`)
+  whose rules are questions put to a System One model. sloplint loads it by
+  name only and exits 2 with an install hint when it is missing. A backend
+  failure under `--judge` is exit 3 and writes no notes. See `docs/JUDGE.md`.
+- `Sloplint::Split`, a paragraph and sentence splitter that keeps offsets into
+  the source, and `Engine.context_window`, the note's context drawn for any
+  span rather than only a regex match. Both are used by the judge.
+- `script/calibrate`, which measures a judge backend against RAID and a
+  current-model side generated from RAID's own prompts, and reports the
+  pass lines from `docs/JUDGE.md`.
+
+### Fixed
+
+- `--markdown` opens and closes a fenced code block only at the start of a
+  line, at any indent, so a fence under `10. ` or a nested bullet still
+  pairs. A fence quoted inside a sentence (```` ``` ````) used to open a
+  block there, and every fence after it paired wrong for the rest of the file.
+- The splitter also drops indented code blocks (four spaces or a tab), lines
+  that are one HTML tag, and YAML front matter under `--markdown`, so none of
+  them reach the judge as prose.
 
 ## [0.8.1] - 2026-09-20
 

data/README.md

@@ -4,6 +4,8 @@ A dependency-free CLI that scans prose for the tells of AI-generated **slop** an
 
 The primary reader is an agent (Claude Code and friends) that runs sloplint, reads the JSON, and rewrites what it flags. Humans are the secondary reader, and everything is built to keep the false-positive rate low enough that a flag is worth trusting.
 
+For the tells a regex cannot see, there is a second gem, [sloplint-judge](#sloplint-judge), whose rules are questions put to a model and whose notes come back in the same JSON. It is optional, it needs an API key, and it is the only part of sloplint that sends your text anywhere.
+
 ## What it catches, and what it doesn't
 
 A pattern earns a place in the catalog only if it shows up constantly in AI writing and rarely in careful human writing. Passive voice, weak adverbs, wordiness, clichés a person reaches for too: those belong in `proselint` or `write-good`, not here. sloplint is not a general prose linter and never tries to be. It hunts the specific fingerprints of a language model, so an agent can act on a flag instead of second-guessing it.
@@ -118,9 +120,13 @@ Flags:    No fluff, no filler, no jargon.
 Does not: No parking on Sundays.
 ```
 
+### `--judge`
+
+`sloplint check --judge` adds the rules of [sloplint-judge](#sloplint-judge) to the run, questions put to a model rather than regexes, and merges the notes into the same array in document order. It needs the sloplint-judge gem and an API key; the section below covers both.
+
 ## The note
 
-One match is one note. JSON output is an array of these, or an object keyed by path when more than one file is scanned. The schema is the contract:
+One match is one note. JSON output is an array of these, or an object keyed by path when more than one file is scanned. Under `--judge` that array or object sits under a `notes` key next to a `judge` key with the backend name, request count and token counts (see [sloplint-judge](#sloplint-judge)). The schema is the contract:
 
 ```json
 {
@@ -144,16 +150,95 @@ One match is one note. JSON output is an array of these, or an object keyed by p
 
 ## Exit codes
 
-Three codes carry the contract. A crash exits nonzero on its own.
+Four codes carry the contract. A crash exits nonzero on its own.
 
 | code | meaning |
 |------|---------|
 | 0    | ran, no notes |
 | 1    | ran, notes found |
 | 2    | bad arguments or usage error |
+| 3    | `--judge` only: the model could not be reached, no notes written |
 
 An unknown id or category in `--select`/`--ignore` is a usage error (exit 2, naming the id) rather than a silent no-op, so a typo can't masquerade as a clean scan. Input that is empty or only whitespace is exit 2 for the same reason: a pipe that delivered nothing must not read as a clean draft. Only when every source is empty — one empty file among several named ones is taken as deliberate.
 
+## sloplint-judge
+
+A second gem in this repository, for the tells a regex cannot see. Its rules are questions put to a System One model (Jev, from TypeSafe) about one paragraph or one sentence at a time: does this paragraph end on a summary, a moral or a hope, does this sentence tell the stated reader anything they did not know, does it name anything a reader could check. The answers come back as sloplint notes, same fields, same JSON, same exit codes, so anything that already reads sloplint's output reads the judge's without change.
+
+One thing is different from the rest of sloplint: the judge sends your text to an API. sloplint on its own never leaves the machine. Every paragraph the judge examines goes to `api.typesafe.ai` over HTTPS, and nothing goes anywhere until you set a key, so the plain `sloplint check` stays offline whether or not the judge is installed.
+
+### Install
+
+```bash
+gem install sloplint sloplint-judge
+sloplint-judge key set        # stores your TypeSafe API key in the OS keychain; it prompts for it
+```
+
+`key set` hands your terminal to the keychain tool (`security` on macOS, `secret-tool` from libsecret on Linux), which asks for the key with echo off, so the key is never on a command line, in shell history or in a dotfile. Setting `TYPESAFE_API_KEY` in the environment works too and takes precedence. One thing the keychain does not do: it keeps the key out of the agent's environment, not out of your account, since any process running as you can read the item back. `sloplint-judge status` says whether a key was found and where, without printing it, and `sloplint-judge key unset` removes the item.
+
+Requires Ruby 3.3+ and sloplint 0.9 or later. The judge is not a plugin of its own: the Claude Code plugin at the root of this repository already carries it, and the `/sloplint:check` skill asks before it runs the judge. A key in the environment makes the judge possible; it does not make it run. The skill puts the question once per conversation, says what leaves the machine and what it costs, and stays offline unless the answer is yes or the request already asked for the judge.
+
+### Run
+
+The one command to know, and the one an agent should use:
+
+```bash
+sloplint check --judge --markdown -o json draft.md
+```
+
+That runs both catalogs and merges the notes in document order. Because the judge spent money, the JSON says how much: the notes sit under `notes` and a `judge` object carries the backend, the number of requests, the token counts the backend reported and the cost in dollars. The same figures go to stderr in one line for the human formats.
+
+```json
+{
+  "notes": [ ... ],
+  "judge": { "backend": "jev-latest", "requests": 9, "input_tokens": 14200, "output_tokens": 610, "cost_usd": 0.000596 }
+}
+```
+
+Without the sloplint-judge gem it exits 2 and says to install it. Without a key it exits 2 and says which variable to set. If the model cannot be reached, or answers in a shape the judge does not understand, it exits 3 and writes no notes at all, the regex ones included, so a partial run can never pass as a clean one.
+
+The gem also puts a `sloplint-judge` executable on your path for the judge on its own:
+
+```
+sloplint-judge [-o full|json] [--register TEXT] [--backend NAME] [command] [args]
+
+check         scan paths (or stdin) with the judge's rules only [default]
+compare A B   which of two passages a plain-prose editor keeps (--drift for rewrites)
+rules         list the judge's rule catalog (add --json)
+explain ID    print one rule's question, levels, rationale and fixtures
+status        say whether a run could happen here, and where the key is, without reading it
+key set       store the backend's key in the OS keychain (the keychain tool prompts for it)
+key unset     remove it from the OS keychain
+version       print the sloplint-judge version
+```
+
+`check` takes `--markdown`, `--select`, `--ignore` and `--strict` with the same meanings as sloplint's. `--strict` runs the three rules that are off by default, runs the sentence rules on every sentence, rather than only in the paragraphs a paragraph rule flagged or skipped as too short, and keeps the notes the model was not confident about. `--register TEXT` says who the reader is; the default is an engineer on the team reading a design document, and every question is asked on that reader's behalf, so a rule such as `no-news` flags a sentence that reader already knows rather than one anybody would.
+
+### The rules
+
+Fourteen rules in two categories. `sloplint-judge rules` lists them and `sloplint-judge explain ID` prints the question the model is asked, the answer that flags, and the fixtures. The bar is a little different from the regex catalog's: a judge rule ships when a reader shown the flagged unit agrees it should go, whoever wrote it, and how sharply it separates model prose from human prose sets its severity. So `throat-clearing` is `info`, not gone: human abstracts open by announcing the paper, and it is dead weight either way.
+
+- **paragraph** (6): `particulars`, a paragraph that names nothing a reader could check; `wrap-up`, a paragraph that ends on a summary, a moral or a hope; `throat-clearing`, a paragraph that opens by announcing its topic; `self-narration`, a paragraph that signposts the document instead of saying something; `promotional`, a paragraph that praises its subject and measures nothing; `same-weight`, a paragraph that states its guesses and opinions as flatly as its measurements. `same-weight` is off by default; name it in `--select` or pass `--strict`.
+- **sentence** (8): `stock-figure`, a stock figure of speech; `no-news`, a sentence that explains what the stated reader already knows; `names-nothing`, a sentence with no specific noun in it; `ends-on-verdict`, a sentence that ends by grading the fact it just stated; `trailing-gloss`, a sentence that ends on an -ing clause drawing its own moral; `unnamed-authority`, a claim handed to experts, studies or many; `stated-stakes`, a sentence that says something matters and not why; `matched-shape`, a pair or triple built to a rhythm rather than to the content. `stated-stakes` and `matched-shape` are off by default; name them in `--select` or pass `--strict`.
+
+Each note's `confidence` is the lower of the rule's own ceiling and how sure the model was of that answer. A note the model was unsure about is dropped unless you pass `--strict`, the same way sloplint drops its low-confidence rules.
+
+### Cost and configuration
+
+One request per paragraph carries the paragraph questions, and one request per examined sentence carries the sentence questions, about eight in parallel. A 2,000-word document runs in a few seconds. Every run reports requests, tokens and cost, in the JSON under `judge` and on stderr. Jev returns token counts and no price, so the dollar figure is computed from TypeSafe's public price: $42 per billion input tokens, and output tokens are free. At that rate a 2,000-word document costs well under a cent.
+
+Configuration is from the environment, plus the OS keychain for the key:
+
+| variable | default | meaning |
+|---|---|---|
+| `TYPESAFE_API_KEY` | none, required | bearer key sent with every request; from the environment, else the keychain item `key set` wrote |
+| `SYSTEMONE_MODEL` | `jev-latest` | model name |
+| `SYSTEMONE_URL` | `https://api.typesafe.ai/v1/systemone` | endpoint; must be `https` on a `typesafe.ai` host |
+| `SLOPLINT_JUDGE_BACKEND` | `jev` | which adapter to use |
+| `SLOPLINT_JUDGE_CONCURRENCY` | `8` | parallel requests |
+
+The design, the calibration that decides which rules ship, and how to add a backend are in [docs/JUDGE.md](docs/JUDGE.md).
+
 ## The rule catalog
 
 82 rules across nine categories, each named for the rhetorical move the construct makes. `sloplint rules` prints them; `sloplint rules --json` gives an agent the enumerable form.

data/docs/SPEC.md

@@ -96,7 +96,8 @@ dependencies at runtime.
 
 Rationale: rules are regexes; the whole thing is a scanner plus an output
 formatter. A dependency-free `gem install sloplint` is the robust, boring
-choice. The name is free on RubyGems (taken on PyPI, so this also sidesteps the
+choice. The one exception is `check --judge`, which loads the separate
+sloplint-judge gem and calls a model over the network; see `docs/JUDGE.md`. The name is free on RubyGems (taken on PyPI, so this also sidesteps the
 collision). Ships a `sloplint` executable from `exe/` (claude.ai rejects a
 plugin with a top-level `bin/`, and sloplint ships as a Claude Code plugin
 too).
@@ -124,7 +125,7 @@ Rakefile                   # rake spec
 ```
 
 RSpec is a **development** dependency (in the gemspec's `add_development_
-dependency`), so the runtime stays dependency-free.
+dependency`), so the runtime stays dependency-free unless `--judge` is passed.
 
 ## CLI surface
 
@@ -160,13 +161,14 @@ Deliberately left out of v1 (add when a real need shows up, not before):
 
 ## Exit codes
 
-Three codes carry the contract. A crash just exits nonzero on its own.
+Four codes carry the contract. A crash just exits nonzero on its own.
 
 | code | meaning |
 |------|---------|
 | 0    | ran, **no notes** |
 | 1    | ran, **notes found** |
 | 2    | bad arguments / usage error |
+| 3    | `--judge` only: the model backend could not be reached, no notes written |
 
 Empty or whitespace-only input is exit 2, like a mistyped rule id: a scan of
 nothing must not report as a clean scan. The text is tested before
@@ -176,7 +178,9 @@ block still exits 0. Only when every source is empty.
 ## Note (the diagnostic object)
 
 One match = one Note. JSON output is an array of these (or an object keyed by
-path when multiple files are scanned).
+path when multiple files are scanned). Under `--judge` the array or object sits
+under `"notes"` beside a `"judge"` object with the backend name, request count
+and token counts; see `docs/JUDGE.md` "Note".
 
 ```json
 {
@@ -249,7 +253,9 @@ RULES = [
 ```
 
 Adding a rule = appending one entry + one bad and one ok fixture. That's the
-whole extension story. No new files, no plugin system (YAGNI).
+whole extension story. The only thing that plugs in is sloplint-judge, and it
+plugs in by name: `check --judge` does a lazy `require "sloplint/judge"` and
+exits 2 with an install hint when it is missing. See `docs/JUDGE.md`.
 
 ### Why Ruby literals, not JSON/YAML
 
@@ -476,6 +482,21 @@ RSpec (dev dependency), run via `rake spec`.
 - LSP server mode, editor plugins, autofix/rewrite. sloplint *flags*; the agent
   rewrites. Autofix is a separate tool if ever.
 - Non-English. Languages other than English are a v2 conversation.
+- Grammar-frequency tells from the corpus-linguistics literature. Reinhart et
+  al. (PNAS 2025) measure GPT-4o at 2.1 times the human rate for
+  nominalizations and 2.6 times for "that" relative clauses on a subject noun
+  ("a framework that enables real-time analysis"). Both shifts are real in RAID
+  and in Claude Sonnet 5 text generated from RAID's prompts, and neither is a
+  shape a regex can flag. A single that-relative is ordinary English ("a move
+  that is likely to put a dent in its accounts", human BBC news) and human
+  news carries one every 650 words; two in one sentence separate no better,
+  and flagging either pushes a writer toward the nominalization instead.
+  Nominalization is a density, not a sentence shape: five or more suffix nouns
+  in one sentence fires on human abstracts at the 2023 model rate, and 50k
+  words of README prose gave 22 hits, every one a bullet list or feature
+  table. Both belong to a judge that reads a paragraph, not to this catalog.
+  Sentence-initial clausal subjects ("That the cache was stale is not in
+  dispute.") were also tried and are near zero on both sides everywhere.
 - ML/embedding-based detection. This is a regex linter on purpose — fast,
   explainable, zero-dependency. Statistical detection is a different product.
 - Scraped corpora. Platform terms prohibit automated collection, and finding a

data/exe/sloplint

@@ -9,5 +9,6 @@ if (RUBY_VERSION.split(".").map(&:to_i).first(2) <=> [3, 3]) < 0
         "macOS ships Ruby 2.6 at /usr/bin/ruby. Install a current one with `brew install ruby`."
 end
 
-require_relative "../lib/sloplint/cli"
+$LOAD_PATH.unshift(File.expand_path("../lib", __dir__))
+require "sloplint/cli"
 exit Sloplint::CLI.run(ARGV)

data/lib/sloplint/cli.rb

@@ -56,6 +56,10 @@ module Sloplint
       strict = false
       select = nil
       ignore = nil
+      judge = false
+      register = nil
+      backend = nil
+      help = false
       p = OptionParser.new do |o|
         o.banner = "usage: sloplint check [options] [paths...]  (\"-\" or no paths = stdin)"
         o.on("-o", "--output-format FORMAT", %w[full json],
@@ -63,21 +67,143 @@ module Sloplint
         o.on("--markdown", "Skip fenced/inline code spans, HTML comments, and URLs before scanning.") { markdown = true }
         o.on("--select IDS", "Only run these rules (comma-separated rule ids or category names).") { |v| select = v.split(",").map(&:strip) }
         o.on("--ignore IDS", "Skip these rules (comma-separated rule ids or category names).") { |v| ignore = v.split(",").map(&:strip) }
-        o.on("--strict", "Run every rule, including the ones that are off by default.") { strict = true }
+        o.on("--strict", "Run every rule, including the ones that are off by default;",
+             "with --judge, also ask the sentence rules about every sentence, about three times the requests.") { strict = true }
+        o.on("--judge", "Also run sloplint-judge's rules, which ask a model (needs the gem and a key).") { judge = true }
+        o.on("--register TEXT", "With --judge: who the reader is.") { |v| register = v }
+        o.on("--backend NAME", "With --judge: which model adapter to use.") { |v| backend = v }
+        # OptionParser answers -h itself when nobody else does, and it answers
+        # it on the real stdout and ends the process. This command prints to
+        # the out it was given and returns, as every other one does.
+        o.on("-h", "--help", "Show this help.") { out.puts(o.help); help = true }
       end
-      p.order!(argv)
+      # permute!, so a flag written after the path is a flag: `sloplint check
+      # draft.md --judge` reads the way anyone would write it.
+      p.permute!(argv)
+      return 0 if help
 
-      unknown = unknown_rule_refs(select) + unknown_rule_refs(ignore)
+      # The judge is a separate gem that depends on this one. The bare require
+      # goes through the load path: exe/sloplint puts this checkout's lib/ at
+      # the front, so from the plugin tree the judge files beside this one win,
+      # and an installed sloplint-judge gem is found otherwise. Only a missing
+      # judge is the install hint; any other LoadError is a real one.
+      if judge
+        begin
+          require "sloplint/judge"
+        # A Gem::LoadError is an installed sloplint-judge that RubyGems will
+        # not activate beside this sloplint: a conflict, or no version that
+        # fits. It carries no path, and to the reader it is the same thing as
+        # no judge at all, which is how `--help` already reports it.
+        rescue LoadError => e
+          raise unless e.path == "sloplint/judge" || e.is_a?(Gem::LoadError)
+
+          err.puts("sloplint: --judge needs the sloplint-judge gem: gem install sloplint-judge")
+          # An installed judge that RubyGems will not activate is a version
+          # conflict, and the hint above tells the reader to install what
+          # they already have. RubyGems says which versions fell out, so
+          # print that too rather than throwing it away.
+          err.puts("sloplint: #{e.message}") if e.is_a?(Gem::LoadError)
+          return 2
+        end
+      end
+      # Accepted and then dropped, these two read as a judge run that was
+      # never asked for: `sloplint check --register "a lawyer" brief.md` runs
+      # the regex rules and says nothing about the reader it was given.
+      unless judge
+        { "--register" => register, "--backend" => backend }.each do |flag, value|
+          next if value.nil?
+
+          err.puts("sloplint: #{flag} needs --judge: without it no model is asked anything.")
+          return 2
+        end
+      end
+      catalog = judge ? RULES + Judge::RULES : RULES
+
+      unknown = unknown_rule_refs(select, catalog) + unknown_rule_refs(ignore, catalog)
       unless unknown.empty?
         err.puts("sloplint: unknown rule or category: #{unknown.join(", ")}")
-        err.puts("run `sloplint rules` to list them.")
+        # Under --judge the selection is read against both catalogs, and
+        # `sloplint rules` lists only one of them: a judge rule id typed with
+        # a letter wrong is not in the list the hint would send you to.
+        lists = judge ? "`sloplint rules` and `sloplint-judge rules`" : "`sloplint rules`"
+        err.puts("run #{lists} to list them.")
         return 2
       end
 
-      rules = select_rules(select, ignore, strict)
+      rules = select_rules(select, ignore, strict, catalog)
       paths = argv.empty? ? ["-"] : argv
       by_path = paths.reject { |x| x == "-" }.size > 1
 
+      # Only the read is invalid input. An ArgumentError out of a scan or a
+      # judge run is a bug in this code, and it raises like one instead of
+      # sending the reader to look for bad bytes in a file that has none.
+      sources = begin
+        read_sources(paths, err:, stdin:)
+      rescue ArgumentError, Encoding::CompatibilityError => e
+        err.puts("sloplint: invalid input: #{e.message}")
+        return 2
+      end
+      return 2 unless sources
+
+      regex_rules, judge_rules = rules.partition { |r| r.is_a?(Rule) }
+
+      judge_usage = nil
+      # Under --judge the output is the {"notes", "judge"} object whether or
+      # not a judge rule survived selection: a caller that asked for the judge
+      # reads the wrapper, and an empty selection is 0 requests, not a
+      # different output shape. 0 requests also means no key: `--judge
+      # --select em-dash` must run on a machine that has none.
+      #
+      # Before the regex scan, not after it: a missing key or a bad
+      # SYSTEMONE_URL is known without asking anything, and finding it out
+      # after every file has been scanned throws that work away to print the
+      # same message.
+      if judge
+        # With nothing to ask, no key is read: the backend is named and never
+        # built, and `sloplint-judge check` reports an empty selection through
+        # the same helper, so the two commands print one line. Either way an
+        # unknown --backend or a key that is not there is the same usage
+        # error, so one rescue answers for both.
+        begin
+          if judge_rules.empty?
+            judge_usage = Judge::Engine.nothing_asked(backend)
+          else
+            judge_backend = Judge::Backend.load(backend)
+          end
+        rescue ArgumentError => e
+          err.puts("sloplint: --judge: #{e.message}")
+          return 2
+        end
+      end
+
+      all_notes = sources.flat_map do |label, text|
+        Engine.scan(text, rules: regex_rules, markdown:, path: label)
+      end
+
+      if judge_usage
+        err.puts("sloplint: judge #{Judge::Engine.usage_line(judge_usage)}")
+      elsif judge_backend
+        begin
+          judged = Judge::Engine.scan_sources(sources, name: "sloplint: judge", err:, rules: judge_rules, backend: judge_backend,
+                                                       markdown:, register: register || Judge::Engine::DEFAULT_REGISTER, strict:)
+        # Exit 3 withholds the regex notes too: a caller that asked for both
+        # and got one would read it as a clean judge run.
+        rescue Judge::BackendError => e
+          err.puts("sloplint: judge backend failure: #{e.message}")
+          return 3
+        end
+        order = sources.each_with_index.to_h { |(label, _), i| [label, i] }
+        all_notes = (all_notes + judged.notes).sort_by.with_index { |n, i| [order[n.path], n.line, n.column, i] }
+        judge_usage = judged.usage
+      end
+
+      emit(all_notes, opts[:format], out:, by_path:, judge: judge_usage)
+      all_notes.empty? ? 0 : 1
+    end
+
+    # Read every path (or stdin for "-") as UTF-8. Returns [[label, text], ...]
+    # or nil after writing the error, so the caller exits 2.
+    def read_sources(paths, err:, stdin: $stdin, name: "sloplint")
       sources = []
       paths.each do |path|
         # Read as UTF-8 whatever the locale says. A sandbox with no LANG set
@@ -90,11 +216,20 @@ module Sloplint
             stdin.read.force_encoding(Encoding::UTF_8)
           else
             unless File.file?(path)
-              err.puts("sloplint: no such file: #{path}")
-              return 2
+              err.puts("#{name}: no such file: #{path}")
+              return nil
             end
             File.read(path, encoding: Encoding::UTF_8)
           end
+        # Prose that is not valid UTF-8 is an invalid input, said here rather
+        # than left to whatever reads the text first: a scan raises on the
+        # first regex, and the judge sends the text to a backend, where it
+        # would be a JSON error after the request was paid for.
+        unless text.valid_encoding?
+          err.puts("#{name}: invalid input: #{path == "-" ? "stdin" : path} is not valid UTF-8")
+          return nil
+        end
+
         sources << [path == "-" ? "-" : path, text]
       end
 
@@ -104,29 +239,19 @@ module Sloplint
       # mistyped rule id sets, and it exits 2 for the same reason.
       if sources.all? { |_, text| text.strip.empty? }
         names = sources.map { |label, _| label == "-" ? "stdin" : label }
-        err.puts("sloplint: empty input: nothing to check in #{names.join(", ")}")
-        return 2
-      end
-
-      all_notes = sources.flat_map do |label, text|
-        Engine.scan(text, rules:, markdown:, path: label)
+        err.puts("#{name}: empty input: nothing to check in #{names.join(", ")}")
+        return nil
       end
+      sources
+    end
 
-      case opts[:format]
-      when "json"
-        out.puts(Output.format_json(all_notes, by_path:))
+    def emit(notes, format, out:, by_path:, judge: nil)
+      if format == "json"
+        out.puts(Output.format_json(notes, by_path:, judge:))
       else
-        text = Output.format_human(all_notes)
+        text = Output.format_human(notes)
         out.puts(text) unless text.empty?
       end
-
-      all_notes.empty? ? 0 : 1
-    # Invalid UTF-8 reaches this two ways: String#strip in the empty check
-    # raises Encoding::CompatibilityError, the engine's regexes raise
-    # ArgumentError. Both are the same thing to the reader.
-    rescue ArgumentError, Encoding::CompatibilityError => e
-      err.puts("sloplint: invalid input: #{e.message}")
-      2
     end
 
     # ── rules ───────────────────────────────────────────────────────────────
@@ -137,19 +262,26 @@ module Sloplint
         o.on("--json", "Emit the catalog as JSON for machine enumeration.") { as_json = true }
       end.order!(argv)
 
-      if as_json
-        payload = RULES.map do |r|
+      render_rules(RULES, json: as_json, out:)
+      0
+    end
+
+    # The catalog as a table or as JSON. Shared with sloplint-judge, whose
+    # rules also carry a unit.
+    def render_rules(catalog, json:, out:)
+      if json
+        payload = catalog.map do |r|
           { id: r.id, category: r.category, severity: r.severity, confidence: r.confidence,
             message: r.message, rationale: r.rationale, suggestion: r.suggestion }
+            .merge(r.respond_to?(:unit) ? { unit: r.unit } : {})
         end
         out.puts(JSON.pretty_generate(payload))
       else
-        RULES.each do |r|
+        catalog.each do |r|
           off = r.confidence == "low" ? " [off by default]" : ""
           out.puts("#{r.id.ljust(24)} #{r.category.ljust(18)} #{r.severity.ljust(8)} #{r.confidence.ljust(7)} #{r.message}#{off}")
         end
       end
-      0
     end
 
     # ── explain ID ────────────────────────────────────────────────────────
@@ -190,10 +322,10 @@ module Sloplint
     # ── helpers ─────────────────────────────────────────────────────────────
     # Ids/categories in refs that match no rule in the catalog. nil (no --select
     # or --ignore given) passes through as no unknowns.
-    def unknown_rule_refs(refs)
+    def unknown_rule_refs(refs, catalog = RULES)
       return [] unless refs
 
-      known = RULES.flat_map { |r| [r.id, r.category] }.uniq
+      known = catalog.flat_map { |r| [r.id, r.category] }.uniq
       refs - known
     end
 
@@ -201,15 +333,15 @@ module Sloplint
     # excludes low-confidence rules unless they are explicitly selected. A
     # category ref selects only that category's non-low rules unless --strict
     # is set; naming a rule by its own id still selects it whatever its
-    # confidence.
-    def select_rules(select, ignore, strict = false)
+    # confidence. catalog is RULES, or RULES plus the judge's under --judge.
+    def select_rules(select, ignore, strict = false, catalog = RULES)
       runs_by_default = ->(r) { r.confidence != "low" }
       rules = if select
-        RULES.select { |r| select.include?(r.id) || (select.include?(r.category) && (runs_by_default.call(r) || strict)) }
+        catalog.select { |r| select.include?(r.id) || (select.include?(r.category) && (runs_by_default.call(r) || strict)) }
       elsif strict
-        RULES
+        catalog
       else
-        RULES.select(&runs_by_default)
+        catalog.select(&runs_by_default)
       end
       if ignore
         rules = rules.reject { |r| ignore.include?(r.id) || ignore.include?(r.category) }
@@ -217,9 +349,37 @@ module Sloplint
       rules
     end
 
-    def global_parser(opts, out:)
-      OptionParser.new do |o|
-        o.banner = <<~BANNER
+    # The judge's part of the agent recipe, which depends on whether the gem
+    # is here. Looked up on the load path without loading it: help must not
+    # pull a model adapter in. The text says the one thing an agent must know
+    # before the flag: the judge sends the text to an API, so ask the person.
+    def judge_recipe
+      # Installed as a gem, the judge is not on the load path until RubyGems
+      # activates it, so the load path alone reports a missing gem. A gem on
+      # disk that asks for a different sloplint cannot be activated beside
+      # this one, and advertising it would send the agent into a conflict.
+      installed = $LOAD_PATH.resolve_feature_path("sloplint/judge") ||
+                  Gem::Specification.find_all_by_name("sloplint-judge").any? { |spec|
+                    spec.dependencies.find { |d| d.name == "sloplint" }&.match?("sloplint", Sloplint::VERSION)
+                  }
+      unless installed
+        return "# sloplint-judge (not installed here) adds model-backed rules for what a regex cannot see: gem install sloplint-judge\n"
+      end
+
+      <<~RECIPE
+        # With the judge (sloplint-judge is installed here). It sends the text to api.typesafe.ai,
+        # so an agent asks the person before running it. Check first, ask, then run:
+        sloplint-judge status                            # exit 0 = a key is set up (reads no key), 2 = not
+        sloplint check --judge --markdown -o json FILE    # output becomes {"notes": [...], "judge": {requests, tokens, cost_usd}}
+        # exit 3 = the model could not be reached; nothing was checked, not even the regex rules
+      RECIPE
+    end
+
+    # The banner names whether the judge gem is here, which costs a scan of
+    # the installed gems. Only --help reads it, so it is built when it is
+    # read and never on the way to a scan.
+    def global_banner
+      <<~BANNER
           sloplint — flag the rhetorical tics and puffery that mark AI-generated prose.
 
           # Recommended for agents:
@@ -228,17 +388,23 @@ module Sloplint
           # each note: {path,line,column,severity,confidence,rule,category,message,excerpt,context,rationale,suggestion,count}
           # (count is present only for the rules that tally items)
 
+          #{judge_recipe}
           usage: sloplint [-o full|json] [command] [args]
 
           commands:
             check        scan paths (or stdin) for AI-slop tells and report notes [default]
                          a first argument that is not a command name is taken as a path
+                         --judge adds sloplint-judge's model-backed rules when that gem is installed
             rules        list the rule catalog (add --json for the machine-readable form)
             explain ID   print one rule's message, rationale, and a bad/ok example
             version      print the sloplint version
 
           global options:
-        BANNER
+      BANNER
+    end
+
+    def global_parser(opts, out:)
+      parser = OptionParser.new do |o|
         o.on("-o", "--output-format FORMAT", %w[full json],
              "Output format: 'full' (human-readable text) or 'json' (default: full).") do |v|
           opts[:format] = v
@@ -254,6 +420,8 @@ module Sloplint
         o.separator ""
         o.separator "See `sloplint explain <id>` for any rule, or docs/SPEC.md for the JSON contract."
       end
+      parser.define_singleton_method(:banner) { CLI.global_banner }
+      parser
     end
   end
 end

data/lib/sloplint/engine.rb

@@ -59,14 +59,21 @@ module Sloplint
     # over blanked text shows code and URLs as a run of spaces. Offsets here are
     # character offsets (MatchData#begin), matching the char-based line_starts_for.
     def context_for(source, match)
-      return "[#{match[0].gsub(/\s+/, " ").strip}]" if match[0].length >= CONTEXT_CHARS
+      context_window(source, match.begin(0), match.end(0))
+    end
+
+    # The same window for any span [b, e) of source, so a tool that locates a
+    # sentence rather than a regex match (the judge) draws its context the
+    # way sloplint does.
+    def context_window(source, b, e)
+      span = source[b...e]
+      return "[#{span.gsub(/\s+/, " ").strip}]" if span.length >= CONTEXT_CHARS
 
-      b, e = match.begin(0), match.end(0)
       pre  = source[[b - CONTEXT_CHARS, 0].max...b]
       post = source[e, CONTEXT_CHARS].to_s
       pre  = "…#{pre.sub(/\A\S*\s+/, "")}" if b > CONTEXT_CHARS
       post = "#{post.sub(/\s+\S*\z/, "")}…" if e + CONTEXT_CHARS < source.length
-      "#{pre}[#{match[0]}]#{post}".gsub(/\s+/, " ").strip
+      "#{pre}[#{span}]#{post}".gsub(/\s+/, " ").strip
     end
 
     # 1-indexed line and column for a char offset into text. Binary-searches a
@@ -101,8 +108,28 @@ module Sloplint
     # with one alternation, so whichever construct opens first is the one that
     # gets consumed: a `<!--` quoted inside backticks is inline code, and a
     # backtick inside a comment is part of the comment.
+    # A URL ends before the punctuation that ends the sentence it sits in:
+    # "See https://example.com. Then do X." is two sentences, and swallowing
+    # the first period would make it one. A URL that really ends in one of
+    # these, a Wikipedia link closing on a bracket, loses that character to
+    # the sentence instead; the text is only being blanked, so what it costs
+    # is one visible character, not a broken link.
+    # A fence opens and closes at the start of a line, with three or more
+    # backticks; the opener may carry an info string and the closer nothing
+    # but whitespace. The line start is the shape that matters: a fence
+    # quoted inside a code span, which is how a document explains fences,
+    # opened a block in the middle of a sentence and every fence after it in
+    # the file paired with the wrong one.
+    #
+    # Any indent is allowed, because CommonMark measures a fence's indent
+    # from its container and a fence under "10. " or a nested bullet stands
+    # further in than three spaces. What that costs is an indented code block
+    # whose own content has a line of backticks in it, which is rare, and the
+    # only thing it costs there is more blanking.
+    MARKDOWN_NOISE = /(?<block>^[ \t]*`{3,}[^\n]*\n.*?^[ \t]*`{3,}[ \t]*$|<!--.*?-->)|(?<inline>`[^`\n]*`|https?:\/\/\S*[^\s.,;:!?)\]])/m
+
     def blank_markdown(text)
-      text.gsub(/```.*?```|<!--.*?-->|`[^`\n]*`|https?:\/\/\S+/m) { |s| s.gsub(/[^\n]/, " ") }
+      text.gsub(MARKDOWN_NOISE) { |s| s.gsub(/[^\n]/, " ") }
     end
   end
 end

data/lib/sloplint/output.rb

@@ -20,13 +20,17 @@ module Sloplint
     end
 
     # JSON: an array of notes, or an object keyed by path when >1 file was scanned.
-    def format_json(notes, by_path: false)
-      if by_path
-        grouped = notes.group_by(&:path).transform_values { |ns| ns.map { |n| note_hash(n) } }
-        JSON.pretty_generate(grouped)
+    # judge: when the judge ran, its backend name and usage; the notes then
+    # sit under "notes" and the usage under "judge", so a caller who asked for
+    # the model's opinion also gets what it cost.
+    def format_json(notes, by_path: false, judge: nil)
+      payload = if by_path
+        notes.group_by(&:path).transform_values { |ns| ns.map { |n| note_hash(n) } }
       else
-        JSON.pretty_generate(notes.map { |n| note_hash(n) })
+        notes.map { |n| note_hash(n) }
       end
+      payload = { "notes" => payload, "judge" => judge } if judge
+      JSON.pretty_generate(payload)
     end
 
     def note_hash(note)

data/lib/sloplint/split.rb

@@ -0,0 +1,348 @@
+# frozen_string_literal: true
+
+require_relative "rules"
+require_relative "engine"
+
+module Sloplint
+  # Splits a document into paragraphs and sentences, keeping each one's offset
+  # and length in the original text so a note can land on the file as written.
+  #
+  # The regex engine does not use this. It exists for the judge and for any
+  # other tool that asks questions about a paragraph or a sentence rather than
+  # matching a pattern across the whole text. See docs/JUDGE.md "Splitting".
+  module Split
+    # text is the span as the file has it, which is what a note quotes.
+    # asked is the same span with what --markdown skips taken out, which is
+    # what the model is shown. They are the same string without --markdown.
+    Sentence = Data.define(:offset, :length, :text, :asked)
+    Paragraph = Data.define(:offset, :length, :text, :asked, :sentences)
+
+    # Furniture: a line that is not prose, whatever a regex would think.
+    # Headings, list items, table rows, block quotes, horizontal rules,
+    # reference-style link definitions and a lone image or link line.
+    FURNITURE = /\A[ \t]*(?:\#{1,6}[ \t]|[-*+][ \t]|\||>|[-*_]{3,}[ \t]*\z|\[[^\]]+\]:[ \t]|!\[)/
+
+    # An ordered-list item, which is furniture too but only sometimes: see
+    # ordered_item?.
+    ORDERED = /\A[ \t]*(\d+)[.)][ \t]/
+
+    # Abbreviations whose trailing period does not end a sentence. Case
+    # matters: "No." is an abbreviation, "no." is the end of a sentence.
+    ABBREV = /\b(?:e\.g|i\.e|vs|etc|cf|Fig|No|Dr|Mr|Mrs|Ms|St|Inc|Ltd|Sec|Ch|Vol)\.\z/
+
+    # The other period that does not end a sentence, matched on its shape
+    # rather than on a list of words: letters written one at a time with a
+    # period after each, which is "U.S.", "U.K.", "e.g.", "i.e." and the rest
+    # of them. Two pairs at least, so the last letter of a word is not one:
+    # "bigger than AT&T." and "a free upgrade from 512K." still end their
+    # sentences. The run is bounded, so a piece is read in one pass.
+    #
+    # One period of this shape is left alone: a single capital standing as a
+    # word. It is an initial in "J. K. Rowling wrote the book.", which is
+    # split into three sentences here, and it is a whole sentence in "Service
+    # A. Service B handles the rest.", which a design document writes far more
+    # often than it names anyone by initial.
+    #
+    # What the run costs instead: a sentence that really ends in one of these
+    # joins the one after it, so "It was in the U.S. It cost $2." is read as
+    # one sentence.
+    INITIALS = /(?<![[:alpha:]])(?:[[:alpha:]]\.){2,6}\z/
+
+    # A number and a period with nothing else between two boundaries is an
+    # enumerator, not a sentence: the "2." of a list item that CommonMark
+    # keeps inside the paragraph above it, because a list may interrupt a
+    # paragraph only when it starts at 1. The whole piece has to be the
+    # number, so a year still closes the sentence it sits in -- "It was
+    # rewritten in\n2021. Then it shipped." is two sentences.
+    ENUMERATOR = /\A\d+\.\z/
+
+    # The judge's view of Markdown differs from the regex engine's in one
+    # way. Fenced code and comments become spaces, as there, but an inline
+    # code span or a URL becomes a run of this character, same length: it is
+    # a word in its sentence, so "`x` runs fast." starts at the backtick and
+    # "Done. `x` runs." is two sentences. Blanked to spaces, the first lost
+    # its head and the second merged. It is a private-use codepoint, which
+    # ordinary prose cannot contain, so a line of real text is never taken
+    # for a blanked one -- a line reading "XXX" used to be.
+    INLINE = "\uE000"
+
+    # A sentence ends at .!? (with an optional closing quote or bracket)
+    # followed by whitespace and a capital, digit, opening quote/bracket, or
+    # a code span, which starts a sentence as any other word does.
+    BOUNDARY = /(?<=[.!?]|[.!?]["'”’)\]])\s+(?=["'“‘(\[A-Z0-9#{INLINE}])/
+
+    # The split keeps its separators, so one of the blocks it hands back is
+    # the break itself. Both of these are written out here rather than inside
+    # the walk, where the interpolation would build the same pattern again for
+    # every block of every document.
+    BREAK_ONLY = /\A#{PARA_BREAK}\z/
+    KEEP_BREAK = /(#{PARA_BREAK})/
+    KEEP_BOUNDARY = /(#{BOUNDARY})/
+
+    module_function
+
+    # text: the source. markdown: blank code, HTML comments and URLs before
+    # splitting, and blank furniture lines so they neither count as prose nor
+    # take the prose around them with them. Splitting happens on the blanked
+    # copy; the texts returned are cut from the original at the same offsets,
+    # so an excerpt is always a string that is in the file. Blanking is
+    # character for character, so the offsets agree.
+    def paragraphs(text, markdown: false)
+      scan, asked = markdown ? blank_furniture(blank(text), blank_blocks(text)) : [text, text]
+      # A document with nothing to leave out is asked about as written, and
+      # then every text is cut once instead of twice.
+      asked = text if asked == text
+      written = Cursor.new(text)
+      shown = asked.equal?(text) ? nil : Cursor.new(asked)
+      out = []
+      at = 0
+      # Split keeping the separators, and walk the offsets arithmetically:
+      # String#index with a start position rescans from 0 on non-ASCII text.
+      scan.split(KEEP_BREAK).each do |block|
+        start = at
+        at += block.length
+        next if block.match?(BREAK_ONLY)
+
+        # Both ends trimmed by the same measure. String#strip takes a leading
+        # NUL off as well as whitespace and /\A\s*/ does not, so a block that
+        # starts with one used to put every offset in the paragraph one
+        # character early and cut its last character off.
+        from_lead = block.lstrip
+        lead = block.length - from_lead.length
+        body = from_lead.rstrip
+        next if body.empty?
+
+        offset = start + lead
+        # The paragraph is cut out of the original once, and its sentences are
+        # cut out of that: cutting each of them out of the whole document by
+        # character offset is what made the walk take the square of the
+        # document's size.
+        para = written.cut(offset, body.length)
+        para_asked = shown ? shown.cut(offset, body.length) : para
+        sents = sentences(body, base: offset, text: para, asked: para_asked)
+        # A one-sentence block ending in a colon is the lead-in to whatever
+        # follows (a code block, a list), not a paragraph.
+        next if markdown && sents.size == 1 && body.end_with?(":")
+
+        unless sents.empty?
+          flat = squash(para)
+          out << Paragraph.new(offset:, length: body.length, text: flat, sentences: sents,
+                               asked: para.equal?(para_asked) ? flat : squash(para_asked))
+        end
+      end
+      out
+    end
+
+    # Sentences of one paragraph. base is where the paragraph starts in the
+    # document, which each sentence's offset is counted from. text is the
+    # paragraph as the file has it and asked is the copy the model is shown,
+    # both cut at the paragraph's own offsets; a sentence is cut out of them
+    # and its whitespace collapsed, so hard-wrapped lines read as one. A
+    # caller that hands over a bare paragraph and neither copy gets the body
+    # itself as both.
+    def sentences(body, base: 0, text: nil, asked: nil)
+      text ||= body
+      asked ||= text
+      # The same cursors as a paragraph is cut with, for the same reason: a
+      # document whose sentences all sit in one paragraph is one long string
+      # to cut them out of.
+      written = Cursor.new(text)
+      shown = asked.equal?(text) ? nil : Cursor.new(asked)
+      cut = lambda do |at, length|
+        flat = squash(written.cut(at, length))
+        Sentence.new(offset: base + at, length:, text: flat,
+                     asked: shown ? squash(shown.cut(at, length)) : flat)
+      end
+      parts = []
+      buf_start = nil
+      pos = 0
+      body.split(KEEP_BOUNDARY).each do |piece|
+        at = pos
+        pos = at + piece.length
+        next if piece.match?(/\A\s+\z/)
+
+        buf_start ||= at
+        # The abbreviation ends where the piece does, and a piece ends at a
+        # .!? that a boundary follows, so it is always inside this piece: the
+        # whole buffer never has to be cut out of the body to see it.
+        next if piece.match?(ABBREV) || piece.match?(INITIALS) || piece.match?(ENUMERATOR)
+
+        parts << cut.call(buf_start, pos - buf_start)
+        buf_start = nil
+      end
+      parts << cut.call(buf_start, pos - buf_start) if buf_start
+      parts
+    end
+
+    def squash(s) = s.gsub(/\s+/, " ").strip
+
+    def blank(text)
+      text.gsub(Engine::MARKDOWN_NOISE) do |s|
+        Regexp.last_match[:inline] ? INLINE * s.length : s.gsub(/[^\n]/, " ")
+      end
+    end
+
+    # The copy the model is shown: fenced code and HTML comments go, because
+    # --markdown says they do, and a code span or URL stays as written,
+    # because it is a word of its sentence and the model has to read it.
+    def blank_blocks(text)
+      text.gsub(Engine::MARKDOWN_NOISE) do |s|
+        Regexp.last_match[:inline] ? s : s.gsub(/[^\n]/, " ")
+      end
+    end
+
+    # Blank furniture lines to same-length spaces, and the indented lines that
+    # continue a blanked one: a wrapped bullet is one bullet, and its second
+    # line is not a sentence of its own. A line that is only code spans or
+    # URLs (a command on a line of its own, a bare link) is furniture too.
+    LONE_INLINE = /\A[ \t]*#{INLINE}[ \t#{INLINE}]*\z/
+    # The same line with sentence punctuation after it. A URL stops before
+    # that punctuation now, so the full stop on a bare link line is written
+    # there rather than part of the link.
+    LONE_INLINE_ENDED = /\A[ \t]*#{INLINE}[ \t#{INLINE}]*[.,;:!?)\]]+\z/
+
+    # A bullet. FURNITURE matches one too, among everything else it matches;
+    # this is here because a bullet and an ordered item are the only two
+    # kinds of furniture a continuation line can belong to.
+    BULLET = /\A[ \t]*[-*+][ \t]/
+    # The second line of a list item, indented under the first.
+    CONTINUED = /\A(?:[ ]{2,}|\t)\S/
+    # An indented code block: four spaces or a tab. Two spaces are still
+    # prose, so an indented paragraph under a heading is kept, which is what
+    # CommonMark says as well -- a code block starts at four.
+    CODE_INDENT = /\A(?:[ ]{4}|\t)/
+    # A line that is one HTML tag, opening or closing, matched on that shape
+    # rather than on a list of element names: <details>, <div>, <br/> and
+    # whatever else a document drops into Markdown all look the same from
+    # here. A sentence that names a tag in running text does not look like
+    # this: "<p> is the tag for a paragraph." has words after the ">".
+    HTML_LINE = %r{\A[ \t]{0,3}</?[A-Za-z][^\n]*>[ \t]*\z}
+    # The --- that opens YAML front matter. FURNITURE reads it as a
+    # horizontal rule wherever it appears; front matter is the block it
+    # opens, and only on the first line of the file.
+    FRONT = /\A---[ \t]*\z/
+
+    # How many lines of YAML front matter the document opens with: the ---
+    # on the first line, everything to the next --- line, and that line. Zero
+    # for a document that does not open with one, so a --- between two
+    # paragraphs stays the horizontal rule it is.
+    def front_matter(lines)
+      return 0 unless lines.first&.chomp&.match?(FRONT)
+
+      close = lines.drop(1).index { |l| l.chomp.match?(FRONT) } or return 0
+
+      close + 2
+    end
+
+    # Both copies at once, line by line: the furniture is decided on the
+    # splitter's copy, where a code span is a placeholder, and the same lines
+    # are blanked in the copy the model is shown. Every line keeps its own
+    # length and its line ending, so an offset means the same thing in the
+    # original and in both copies. The line ending is taken off before the
+    # tests run: on a CRLF file it would otherwise sit between the line and
+    # the \z that a horizontal rule or a bare link ends at, and neither would
+    # be recognised as furniture.
+    def blank_furniture(scan, shown)
+      lines = scan.each_line.to_a
+      front = front_matter(lines)
+      item = false
+      prose = false
+      code = false
+      opens = true
+      pairs = lines.zip(shown.each_line.to_a).each_with_index.map do |(whole, also), i|
+        l = whole.chomp
+        ending = whole[l.length..]
+        blank = l.strip.empty?
+        # Only a list item runs on to the next line. An indented line under a
+        # heading, a table row, a horizontal rule or a link definition is an
+        # indented paragraph, and chaining from those dropped the prose along
+        # with the furniture above it.
+        listed = l.match?(BULLET) || ordered_item?(l, prose)
+        indented = l.match?(CODE_INDENT)
+        # An indented code block opens where a paragraph cannot be running
+        # already -- the first line, or after a blank or furniture line --
+        # and where the indent is not a list item's second line. It then runs
+        # for as long as the indent holds.
+        code = (code && (indented || blank)) || (indented && !item && opens)
+        dropped = i < front || code || listed || l.match?(FURNITURE) || l.match?(HTML_LINE) ||
+                  lone_inline?(l, prose) || (item && l.match?(CONTINUED))
+        item = listed || (item && l.match?(CONTINUED))
+        prose = !dropped && !blank
+        opens = blank || dropped
+        dropped ? ["#{" " * l.length}#{ending}"] * 2 : [whole, also]
+      end
+      [pairs.map(&:first).join, pairs.map(&:last).join]
+    end
+
+    # A line that is only code spans or URLs is furniture: a command on a
+    # line of its own, a bare link. With sentence punctuation after it, only
+    # where no sentence can have been running already -- after a line of
+    # prose it is the wrapped tail of that sentence, and "Run it with\n`make
+    # test`." is one sentence that would otherwise lose its object.
+    def lone_inline?(line, after_prose)
+      return true if line.match?(LONE_INLINE)
+
+      !after_prose && line.match?(LONE_INLINE_ENDED)
+    end
+
+    # CommonMark lets an ordered list interrupt a paragraph only when it
+    # starts at 1. So a hard-wrapped prose line that begins with a year --
+    # "The library was released in\n2019. It was rewritten in\n2021." -- is
+    # prose, not two list items, and the sentences on it stay in view.
+    def ordered_item?(line, after_prose)
+      m = ORDERED.match(line) or return false
+      !after_prose || m[1] == "1"
+    end
+
+    # Cutting a span out of a string by character offset walks the string from
+    # its start, because a character is not a fixed number of bytes, so cutting
+    # every paragraph of a document out of it one at a time takes the square of
+    # the document's size. The spans are asked for in the order they appear, so
+    # a cursor holds the byte offset of the last character cut at and answers
+    # each one in the length of the span itself: byteslice takes a byte offset,
+    # and a byte offset costs nothing to reach.
+    class Cursor
+      def initialize(text)
+        @text = text
+        @char = 0
+        @byte = 0
+      end
+
+      # The span of length characters at character offset at. A cursor only
+      # goes forward: asked to go back it would answer with the wrong span,
+      # and a note would quote a string that is not in the file.
+      def cut(at, length)
+        raise ArgumentError, "a cursor at #{@char} was asked for #{at}" if at < @char
+
+        advance(at - @char)
+        advance(length)
+      end
+
+      private
+
+      def advance(count)
+        return "" unless count.positive?
+
+        s = window(count)
+        @char += s.length
+        @byte += s.bytesize
+        s
+      end
+
+      # A character is at most four bytes in UTF-8, so four bytes per character
+      # is a window wide enough to hold the ones asked for. Any other encoding
+      # widens it until it is, or until the string ends. A character split by
+      # the far edge of the window is past the count asked for, so it is never
+      # one of the ones returned.
+      def window(count)
+        width = count * 4
+        loop do
+          s = @text.byteslice(@byte, width)
+          return s[0, count] if s.length >= count || s.bytesize < width
+
+          width *= 2
+        end
+      end
+    end
+  end
+end

data/lib/sloplint/version.rb

@@ -1,5 +1,5 @@
 # frozen_string_literal: true
 
 module Sloplint
-  VERSION = "0.8.1"
+  VERSION = "0.9.0"
 end

metadata

@@ -1,7 +1,7 @@
 --- !ruby/object:Gem::Specification
 name: sloplint
 version: !ruby/object:Gem::Version
-  version: 0.8.1
+  version: 0.9.0
 platform: ruby
 authors:
 - Benjamin Jackson
@@ -58,6 +58,7 @@ files:
 - lib/sloplint/engine.rb
 - lib/sloplint/output.rb
 - lib/sloplint/rules.rb
+- lib/sloplint/split.rb
 - lib/sloplint/version.rb
 homepage: https://github.com/benjaminjackson/sloplint
 licenses: