@supersuit/hyperspec 0.5.0 → 0.7.0

package/CHANGELOG.md

@@ -1,5 +1,171 @@
 # Changelog
 
+## 0.7.0 (2026-09-29)
+
+A spec can now have its judgments made and recorded. `check` covers what a function of the spec
+and the draft can decide; the rest of a spec's checks are rubrics, and until this release nothing
+ran them. hyperspec still calls no model. `hyperspec judge prepare` writes one packet per judgment
+station, holding the rubric, fixed instructions, the inputs and the exact shape of the answer, for
+an outside judge to fill: your agent, any model, or a person. `hyperspec judge record` checks the
+verdict, derives the station's status from it by a fixed rule, and records it in the runs ledger.
+Every passage a judge quotes must be in the draft, and the two blind tests are scored
+against answer keys the judge never sees. `hyperspec learn` works from the other end: given the
+first draft a factory wrote and the draft a person approved, it lists the edits, has a judge name
+the spec block that should have prevented each one, and names one next move.
+
+**No behavior change for lint or check.** Every 0.6.0 test passes unchanged, no finding id changed,
+and no schema field was added. The runs ledger gains two kinds of line, `judge` and `learn`; check
+reads only its own, so a 0.6 ledger's check verdicts are exactly what they were, and the new lines
+keep lint's test 9 passing.
+
+- `hyperspec judge prepare <spec> --draft <file> --out <dir> [--only a,b] [--force] [--json]`
+  lints the spec first, as `check` does, then writes `<station>.packet.json` for each station that
+  applies, in the order `doctor, lineup, reader, persona, attribution, knowledge`, and prints a
+  skip line with the reason for each that does not. Packets keep the paths as given and are
+  byte-identical for the same files. It refuses to overwrite any file without `--force`, naming
+  each. Exit 0 written, 1 a station crashed, 2 usage, or lint's own 1 or 3.
+- The two blind tests write an answer key beside their packets, `lineup.key.json` and
+  `attribution.key.json`, for a person to read. Hand a judge only the `*.packet.json` files, and
+  each of the two blind packets to its own fresh context, with no access to the draft and apart
+  from the other packets, which carry it: their
+  instructions say to decide from the packet's inputs alone and open no file it names, but a judge
+  that can open the draft can always cheat. `record` never reads a key: it builds it again.
+- `hyperspec judge record <packet> --verdict <file> [--json]` hashes the spec and the draft again
+  and rebuilds the packet from them and from the files the station reads (the DNA goldens, the
+  claims ledger). A file that changed makes the packet stale (`judge-stale`), and a packet that is
+  not those exact bytes is refused (`judge-packet-altered`); either way nothing is recorded. A
+  stale packet has `learn record`'s shape, `{ "invalid": true, "stale": true }` with `--json` and
+  "stale packet, nothing recorded" on the terminal. It must run in the folder `prepare` ran in. Then the verdict is validated, every problem named. Exit
+  0 the station passed, 1 it failed or the verdict was refused, 2 usage.
+- The evidence rule: every span a verdict cites is at least three words and appears in the draft,
+  on whole words, after whitespace runs become one space and curly, low and angle quotation marks
+  and apostrophes become straight ones (primes do not).
+- Six stations. `doctor`: every goal condition, and whether the reader would take
+  `goal.next_if_worked` now. `lineup`: the draft's paragraph nearest the goldens' median length
+  beside up to three goldens' paragraphs, each reflowed to one line and shuffled with a seed
+  derived from the draft's full text, which the packet does not carry, so the packet cannot reveal
+  the order (a draft that is one paragraph and nothing else skips, since its hash would name its
+  candidate); it passes when the judge picks a golden, which a random pick does three times in four.
+  `reader`: where the audience's reader got lost (warnings) or stopped, and whether they would
+  take their own next step. `persona`: every break of stance, assertion, `will_not_say` or an
+  unsourced fact, against the claims ledger's texts (`null` with no ledger, and then no
+  unsourced fact may be reported). `attribution`, fiction only: the speaker of each dialogue line
+  whose speech tag names one, scored per speaker and averaged, passing at 80 percent; a line whose
+  speaker is in doubt (a split quote joins only across exactly one speech tag, and the part after
+  any longer narration is left out), or that repeats a golden or rejected line, is left out and
+  counted, so the key is never wrong (the worked story tests 19 of its 36 lines). `knowledge`, fiction only: every
+  place a character knows something before their timeline gives it to them.
+- Judge ledger lines carry `packet_sha256`, the hash of what the judge was shown, and
+  `inputs_sha256`, the hash of its inputs, and follow check's rules against the last judge line
+  for the same station and draft, saying `judgment` where check says `check` ("no change since the
+  last passing judgment"), with three more: what changed is judged by the packet, naming the
+  draft, the spec, the file the station reads besides them when the inputs changed (the DNA scope,
+  the claims ledger), or the packet's fixed text when only that changed, so adding the golden or
+  the claims a failing station asked for and passing is improved; one-shot needs draft bytes never judged by that
+  station under any name; and improved needs the packet to have changed, since a judge answering
+  differently about the same packet is not the work improving ("the verdict changed; nothing the
+  judge was shown changed").
+- `hyperspec learn prepare <spec> --first <draft> --approved <draft> --out <dir> [--force]`
+  splits both drafts into sentence units (a paragraph's opening heading, each list item and each
+  sentence, with common abbreviations and lower-case continuations joined), diffs them, and
+  writes `learn.packet.json` with every edit as a hunk (`deleted`, `inserted` or `replaced`, with
+  both texts and a sentence count). A hunk never crosses a paragraph or a heading, and reflowed
+  text is no edit. Past 10,000,000 comparisons after the common start and end are set aside, it is
+  a usage error.
+- `hyperspec learn record <packet> --verdict <file>` checks that every hunk is classified once by a
+  block the spec has written, or `none`, counts edits and sentences per block, and names one move
+  for the block with the most sentences, from a fixed table (`dna`: add a golden or a style rule).
+  It appends a `learn` line to the runs ledger, always `not-improved` since the spec has not
+  changed yet, and never edits the spec.
+- The worked examples ship their packets (`essay/judge/`, `story/judge/`), with no key, and one
+  sample verdict per packet (`*/sample-verdicts/`), filled in by hand and marked `sample`. Two
+  of them fail, honestly: the essay's lineup (the draft's passage is the only one resting on a
+  figure) and the story's persona (three process details no claim holds). The essay's goal now
+  names the card its draft ends on as the reader's next step (`goal.next_if_worked`, matching the
+  spec's `agenda-card` decision), so its doctor sample passes. The essay adds a learn pair in `essay/learn/` and a sample verdict whose
+  tally sends the next move to `dna`. A test holds every packet to what `prepare` writes and records
+  every sample.
+- WRITING.md gains "Judging a draft" and "Learning from edits": the commands and exit codes, the
+  packet, the evidence rule, what `record` refuses, each station's rules and findings table (held
+  by a test to the ids the code raises and their kind), the ledger rules, the samples and what they
+  found, sentence units, the diff, the tally and its move table. The quotes station's section
+  now points at `attribution` for fiction's dialogue.
+
+## 0.6.0 (2026-09-29)
+
+A spec can now check a draft. Until this release hyperspec could tell you whether a writing spec
+was ready; it could not tell you whether the piece written from it met the spec. `hyperspec
+check` runs seven deterministic stations against a draft: the length and required parts, the
+terms the reader needs defined, the claims ledger, quotations, private material, the writer's
+measured style, and links. None of them calls a model or touches the network, so the same draft
+and spec always give the same answer. Each run leaves one line in the spec's runs ledger saying
+whether the draft passed first time, improved, or did not, and why.
+
+**No behavior change for lint.** Every spec without `writing.audience.terms` passes and fails
+exactly as it did in 0.5.0, and no lint finding id changed. That field is the one schema
+addition, and it is optional.
+
+- `hyperspec check <spec> --draft <file> [--json] [--only a,b]` needs a writing spec
+  (`profile: writing`) and lints it first: a spec that fails lint, or is blocked on an open
+  decision, runs no station and exits with lint's code. Then it runs every station in a fixed
+  order, `form, terms, claims, quotes, private, dna, links`, and prints each one's `pass`, `fail`
+  with findings, or `skip` with the reason. Warnings print under their station and never fail it.
+  Exit 0 when every station that ran passed, 1 when one failed, 2 on usage (no spec, no
+  `--draft`, an unreadable draft, a spec without the writing profile, an `--only` that names no
+  known station); under `--json` a usage error is one document, `{ spec, draft, error }`.
+  `--only` runs a subset, still in the fixed order. A finding names the draft line it points at
+  where there is one, quotes at most 80 characters of the draft, and never prints an absolute
+  path. A byte order mark at the start of the draft is ignored. A station that throws becomes one
+  failing finding, `station-<name>-crashed`, and the rest still run.
+- `form`: word count against `writing.form.length` (only `unit: words` is measured; another unit
+  skips the station), and every `required_parts` entry present as an ATX heading of that text
+  (indented up to three spaces, closing `#`s allowed), or as a line starting `part:`, outside
+  code blocks.
+- `terms`: every term in the new optional `writing.audience.terms`, other than those in `knows`,
+  is defined at its first appearance: in that sentence or the next, the term followed within six
+  words by `is`, `means`, `refers to` or a colon, or by a parenthesis. A mechanical proxy for a
+  definition, and documented as one. No `terms` list: skip.
+- `claims`: every claim in the JSONL ledger at `writing.sources.ledger` (`text`, `source`,
+  optional `span`) still appears in the draft word for word, and has a real source; unsourced
+  claims warn instead under `unsourced_claim: warn`, and point at the draft line where the claim
+  appears. A missing ledger fails. The ledger is the list of claims: the station does not decide
+  what counts as one.
+- `quotes`: every double-quoted span of four words or more appears word for word in a `quote` or
+  `story` segment of a marked material, never a private one; when the sentence names a quote
+  segment's speaker, by the full name or by its first word (when that word has two or more
+  letters and is not a common function word such as "the"), the span must come from that
+  speaker. A spec with `fiction: true` skips the station, since a character's dialogue is
+  invented rather than quoted.
+- `private`: no run of eight words from a `private` segment appears in the draft. Segments of four
+  to seven words are checked whole; shorter ones are counted in one warning and never quoted.
+- `dna`: with a current `writing.dna.scope_dir`, the draft is measured the way goldens are and
+  each feature compared with the scope's, from v ÷ 1.5 to the larger of v × 1.5 and v + 5; drift,
+  and an em dash where the scope has none (pointing at the first one), are warnings. No scope
+  folder: skip.
+- `links`: inline, reference and bare links are well-formed http, https or mailto (a bare
+  `https://` with no host fails); relative links resolve to a file beside the draft; a `/` link
+  warns, since there is no site root to resolve it against; a full or collapsed reference needs
+  its definition. A bare `[label]` is a link only when its label is defined, so `[sic]`, `[x]` and
+  `[1]` are text. No network access.
+- The runs ledger: each check appends `{ at, kind: "check", draft, draft_sha256, spec_sha256,
+  stations, verdict }` to `improvement.ledger`, with `draft` relative to the spec's folder. A run
+  with `--only` adds `partial: true`, is `not-improved` with the reason `partial run: <stations>`,
+  and is ignored by later verdicts. A full run is compared with the last full line for the same
+  draft: `one-shot` when there is none and every station passes; `improved` when that line failed
+  and every station passes now, with `change` naming exactly the stations that failed then and
+  pass now; otherwise `not-improved`, with a `reason` that says which case it is, such as "no
+  change since the last passing check" or "spec changed; failing stations: terms". The lines keep
+  lint's test 9 passing.
+- New optional field `writing.audience.terms`: a list of real strings when present (test 1,
+  `writing-audience-terms`, naming a scalar that is not a list, an empty list, and each entry that
+  is not a string, is empty or is a placeholder). Absent, nothing changes.
+- The worked examples each ship a draft that passes every station, with its claims ledger:
+  `examples/writing/essay/draft.md` and `examples/writing/story/draft.md`. Both specs now list
+  `audience.terms`, and their `required_parts` are the drafts' headings. The essay's interview
+  quotes now name their speaker `dana, an engineering manager`, and the essay links the survey
+  summary it cites. A test runs `check` on both. WRITING.md gains "Checking a draft": each station, what it cannot check, a findings table
+  per station held by a test to the ids the stations raise, and the ledger line.
+
 ## 0.5.0 (2026-09-29)
 
 Scoped writer DNA. A writer does not have one voice: the same person writes differently for a

package/README.md

@@ -28,6 +28,11 @@ improvement ledger. Every test is defined in [SPEC.md](SPEC.md).
 | `hyperspec segments init <material> --id <mid> [--out F] [--by paragraph\|sentence]` | Split a material into segments to label. Refuses to overwrite an existing file. |
 | `hyperspec dna init <scope-dir> --writer W --form F --audience A --purpose P` | Start a writer-DNA scope folder. Refuses to overwrite an existing `scope.md`. |
 | `hyperspec dna measure <scope-dir>` | Check every golden in a scope and write its measured features. |
+| `hyperspec check <spec> --draft <file> [--only a,b]` | Run a writing spec's deterministic stations against a draft. |
+| `hyperspec judge prepare <spec> --draft <file> --out <dir> [--only a,b] [--force]` | Write one packet per judgment station, for an outside judge to fill. |
+| `hyperspec judge record <packet> --verdict <file>` | Check a judge's verdict against its packet, derive the station's status, and record it. |
+| `hyperspec learn prepare <spec> --first <draft> --approved <draft> --out <dir> [--force]` | Write the edits between a first draft and the approved one, for a judge to classify by spec block. |
+| `hyperspec learn record <packet> --verdict <file>` | Count the classified edits by block and name one next move. |
 | `hyperspec recipe check <output-or-recipe>` | Check that a recipe records everything the standard asks for. |
 | `hyperspec recipe approve <recipe> --by <slug>` | Record who approved the output. |
 | `hyperspec reproduce <recipe> [--restore]` | Re-check every hash the recipe recorded. Never runs a model. |
@@ -42,6 +47,16 @@ Every command except `init`, `segments init` and `dna init` takes `--json`. `hyp
 test fails, 3 when every test passes but a decision is still open (blocked), and 2 on a usage
 error or a file that cannot be read, has broken frontmatter, or is not a hyperspec.
 
+`hyperspec check` exits 0 when every station it ran passed, 1 when one failed, and 2 on a usage
+error, a draft that cannot be read, or a spec without `profile: writing`. A spec that is not ready to check against exits with lint's
+own code, 1 or 3, and no station runs.
+
+`hyperspec judge record` exits 0 when the station passed and 1 when it failed or the verdict was
+refused (invalid, or its packet stale or edited), and `hyperspec learn record` 0 when it recorded
+and 1 when it refused. `judge prepare` and `learn prepare` exit 0 when they wrote their packets and
+with lint's own code when the spec is not ready, and `judge prepare` exits 1 when a station could
+not build its packet. All four exit 2 on a usage error.
+
 The recipe commands use the same numbers: 0 ok, 1 a check failed or the child regressed, 2
 usage or unreadable input, 3 pending, when `regenerate` has stages waiting for a runner.
 `regenerate` also exits 1 when a stage it ran reported a failing verdict, or none. A stage it
@@ -132,7 +147,7 @@ The skeleton fails until every placeholder is real and its four open questions (
 who speaks, who reads, what changes) are answered. The blocks, every field, and which test
 each rule reports under are in [WRITING.md](WRITING.md). Two complete specs that pass with
 nothing to warn ship in `examples/writing/`: an essay for new managers, and a short story with
-two characters whose voices a judge can tell apart.
+two characters whose voices a judge can tell apart. Each comes with a draft written to it.
 
 ### Marking materials
 
@@ -176,6 +191,52 @@ it did in 0.4. The folder shape, every feature, and every finding are in
 [WRITING.md](WRITING.md#scoped-dna); `readScope` and `measureFeatures` are exported from
 `@supersuit/hyperspec/writing`.
 
+### Checking a draft
+
+Once a draft exists, `check` holds it to its spec with seven stations, none of which calls a
+model or touches the network: `form` (length and required parts), `terms` (every word in the new
+optional `writing.audience.terms` is defined where it first appears), `claims` (the claims
+ledger still matches the draft, and every claim has a source), `quotes` (in nonfiction, every quotation of four
+words or more is word for word in a marked quote), `private` (no run of eight words from a
+private segment), `dna` (the draft's measured style beside its scope's, as warnings) and `links`
+(well-formed, and relative links resolve).
+
+```bash
+npx @supersuit/hyperspec check essay.hyperspec.md --draft essay/draft.md
+```
+
+It lints the spec first, prints each station's pass, fail or skip, and appends one line to the
+spec's runs ledger with a verdict. Both examples ship a draft that passes. What each station
+checks and cannot check, and every finding, are in
+[WRITING.md](WRITING.md#checking-a-draft).
+
+### Judging a draft and learning from edits
+
+The rest of a spec's checks are judgments: whether each goal condition holds, where a reader gets
+lost, whether a passage can be told from the writer's goldens, whether the persona holds, and in
+fiction whether the characters' voices can be told apart and whether anyone knows something too
+early. hyperspec never calls a model. `judge prepare` writes a packet per station for an outside
+judge, your agent, any model or a person, and `judge record` checks the verdict (every quoted
+passage must be in the draft, and the blind tests are scored against keys the judge never
+sees), derives the station's status, and records it in the runs ledger.
+
+```bash
+npx @supersuit/hyperspec judge prepare essay.hyperspec.md --draft essay/draft.md --out essay/judge
+npx @supersuit/hyperspec judge record essay/judge/doctor.packet.json --verdict doctor.verdict.json
+```
+
+Hand the judge the `*.packet.json` files only: the answer keys are written beside them. Give the
+two blind packets (`lineup`, `attribution`) to a judge in a fresh context with no access to the
+draft, such as a new conversation, never to an agent working in the draft's folder: a judge that
+can open the draft can always find the answer, whatever the packet tells it. Give each blind
+packet its own context, apart from the other packets too, since those carry the draft. `learn`
+closes the loop from the other end. Given the first draft a factory wrote and the draft a person
+approved, `learn prepare` lists the edits, a judge names the spec block that should have prevented
+each, and `learn record` counts them by block and names one next move, such as "add a golden or a
+style rule". It never edits the spec. Both examples ship their packets and hand-filled sample
+verdicts. The packet shapes, every station's rules and findings, and the learn tally are in
+[WRITING.md](WRITING.md#judging-a-draft).
+
 ## The format
 
 A hyperspec is a markdown file with a YAML frontmatter block: `decisions`, `requirements`,

package/SPEC.md

@@ -97,7 +97,7 @@ examples:
   - path: examples/minimal.hyperspec.md
     why: the smallest spec that passes all nine tests
 resume:
-  next_action: collect adopter issues on 0.5, scoped DNA included, and cut 0.6 from them
+  next_action: collect adopter issues on 0.7, the judge and learn commands included, and cut 0.8 from them
 feedback:
   issues: https://github.com/SupersuitUp/hyperspec/issues
   fork: MIT; fork it for your own purposes and say so in your SPEC
@@ -109,7 +109,7 @@ improvement:
 
 A person writing for another person leaves most of the specification unsaid, because the other person fills the gaps from shared context. An agent has none of that context, so it fills every gap with the average, and the average is what reads as middling. Hyperspecification is writing down the gaps. It is a level of detail that would feel like overkill between two people and is exactly enough for an agent: every decision the agent would otherwise guess is either decided, delegated with the rule for deciding it, or marked open, so the work stops instead of guessing.
 
-**Version 0.5.0** (2026-09-29)
+**Version 0.7.0** (2026-09-29)
 
 ## What makes a spec a hyperspec
 
@@ -220,7 +220,7 @@ Each row lists every condition under which `hyperspec lint` fails that test. A w
 | 8 its adopters can push back on it | `feedback.issues` or `feedback.fork` missing |
 | 9 it improves itself | `improvement.ledger` missing; a ledger path that exists and is not a readable file; if the ledger file exists, a line that is not a JSON object, a `verdict` outside one-shot, improved or not-improved, `improved` without `change`, `not-improved` without `reason`. A declared ledger that does not exist yet is a warning |
 
-A profile adds its own conditions to these rows. The writing profile's are in [WRITING.md](WRITING.md#the-test-mapping), including the checks on each material's segments file: every material marked, every segment labeled from the closed set and matching its material word for word, the marking current (tests 1 and 4), and no spine claim citing a private or question segment (test 5). A writing spec that names a writer-DNA scope folder with `writing.dna.scope_dir` is also checked against it: the scope matches the spec (test 1), every golden has a person's approval and a source (test 4), no golden comes from outside the scope (test 5), and every golden has its why and the scope's measured features are current (test 6). `scope_dir` is optional, and without it nothing changes.
+A profile adds its own conditions to these rows. The writing profile's are in [WRITING.md](WRITING.md#the-test-mapping), including the checks on each material's segments file: every material marked, every segment labeled from the closed set and matching its material word for word, the marking current (tests 1 and 4), and no spine claim citing a private or question segment (test 5). A writing spec that names a writer-DNA scope folder with `writing.dna.scope_dir` is also checked against it: the scope matches the spec (test 1), every golden has a person's approval and a source (test 4), no golden comes from outside the scope (test 5), and every golden has its why and the scope's measured features are current (test 6). `scope_dir` is optional, and without it nothing changes. The writing profile also has an optional `writing.audience.terms`, the words a piece uses that its reader may not know: when present it must list real terms (test 1), and `hyperspec check` reads it to require each term's definition where the draft first uses it. Without it nothing changes.
 
 ## Exit codes
 
@@ -236,7 +236,7 @@ A profile adds its own conditions to these rows. The writing profile's are in [W
 Every run of a skill that works from a hyperspec writes one line to the ledger named in `improvement.ledger`, one JSON object per line, with a `verdict`:
 
 - **one-shot**: no intervention, nothing to learn.
-- **improved**: the skill, the spec template, or a component library changed, and the line carries `change`, naming what changed.
+- **improved**: the skill, the spec template, or a component library changed, and the line carries `change`, naming what changed. On a `kind: "check"` line, which `hyperspec check` writes, it means the draft now passes every station after the last full check of it failed, and `change` names those stations.
 - **not-improved**: nothing changed, and the line carries `reason`, a reason a later session can argue with, such as "the correction was about this piece only" or "the fix belongs to a shipped skill and was filed as an issue".
 
 Silence is not a verdict. A run that learned nothing has to say so and why, and a ledger line with none of the three verdicts fails the ninth test.