@supersuit/hyperspec 0.9.0 → 0.9.1

package/CHANGELOG.md

@@ -1,5 +1,31 @@
 # Changelog
 
+## 0.9.1 (2026-09-30)
+
+Two fixes, each found by the first book written as a sequence.
+
+- The sequence station no longer reads a word inside a longer defined term as a use of the shorter
+  one. A book that defines "Thinking level" in Lesson 2 and "Level" in Lesson 22 failed
+  `station-sequence-used-before-defined` and `station-sequence-quiz-used-before-defined` at every
+  "thinking level" before Lesson 22. Before it looks for a term, the station now blanks every other
+  defined term that contains it as a whole-word phrase, a trailing plural "s" included, in the order
+  guard, the quiz order guard and quiz coverage. The shorter term on its own still counts, and the
+  longer term no longer tests the shorter one in a quiz. `longerTerms` and `maskLonger` are exported
+  from the station.
+- `hyperspec segments init <material> --id <mid> --keep <old>` re-marks an edited material without
+  relabeling what did not change (issue #2). Every segment whose text, trimmed, a segment in `<old>`
+  has keeps that segment's id, label and every other key (`own`, `source`, `teller`, `speaker`, and
+  anything added by hand), with new offsets; the rest start `unlabeled` with the next unused
+  `s<n>` id, and the command lists them. The id is carried so the spine's citations keep pointing
+  at the same words. `--keep` may name the file being written, so a material is re-marked in place;
+  otherwise an existing file is still never overwritten. It exits 2 for a `--keep` file that cannot
+  be read, has a line that is not a JSON object, or marks another material. `carrySegments` in
+  `src/segments.mjs` is the logic.
+
+**Behavior change:** a sequential work that defines a term inside a longer one may now pass where
+0.9.0 failed it, and a quiz whose only question on a shorter term used it inside the longer term now
+fails `station-sequence-quiz-untested` for that term. No finding id changed.
+
 ## 0.9.0 (2026-09-29)
 
 The pressure test a draft gets before it ships is now part of the run, and so is answering it.

package/README.md

@@ -25,7 +25,7 @@ improvement ledger. Every test is defined in [SPEC.md](SPEC.md).
 | `hyperspec lint <file...> [--json]` | Score each hyperspec against the nine tests. |
 | `hyperspec init <file> [--title T] [--kind K]` | Write a new hyperspec skeleton. Refuses to overwrite an existing file. |
 | `hyperspec init <file> --profile writing [--title T] [--form F] [--fiction]` | Write a writing-spec skeleton, every block shown with placeholders. |
-| `hyperspec segments init <material> --id <mid> [--out F] [--by paragraph\|sentence]` | Split a material into segments to label. Refuses to overwrite an existing file. |
+| `hyperspec segments init <material> --id <mid> [--out F] [--by paragraph\|sentence] [--keep OLD]` | Split a material into segments to label. With `--keep`, re-mark an edited material: every segment whose text is unchanged keeps its id and labels, and only the rest are listed to label. Refuses to overwrite an existing file unless `--keep` names it. |
 | `hyperspec dna init <scope-dir> --writer W --form F --audience A --purpose P` | Start a writer-DNA scope folder. Refuses to overwrite an existing `scope.md`. |
 | `hyperspec dna measure <scope-dir>` | Check every golden in a scope and write its measured features. |
 | `hyperspec check <spec> [--draft <file>] [--only a,b]` | Run a writing spec's deterministic stations against a draft, or, for a sequential work, against its files in reading order. |

package/SPEC.md

@@ -109,7 +109,7 @@ improvement:
 
 A person writing for another person leaves most of the specification unsaid, because the other person fills the gaps from shared context. An agent has none of that context, so it fills every gap with the average, and the average is what reads as middling. Hyperspecification is writing down the gaps. It is a level of detail that would feel like overkill between two people and is exactly enough for an agent: every decision the agent would otherwise guess is either decided, delegated with the rule for deciding it, or marked open, so the work stops instead of guessing.
 
-**Version 0.9.0** (2026-09-29)
+**Version 0.9.1** (2026-09-30)
 
 ## What makes a spec a hyperspec
 

package/WRITING.md

@@ -397,14 +397,15 @@ writing spec cannot pass until every material it draws on is marked.
 npx @supersuit/hyperspec segments init materials/voice-memo.md --id voice-memo
 ```
 
-`hyperspec segments init <material> --id <mid> [--out <file>] [--by paragraph|sentence]` writes
-`<material>.segments.jsonl`, or the path `--out` names. `--by paragraph`, the default, makes one
+`hyperspec segments init <material> --id <mid> [--out <file>] [--by paragraph|sentence] [--keep <old>]`
+writes `<material>.segments.jsonl`, or the path `--out` names. `--by paragraph`, the default, makes one
 segment per paragraph. `--by sentence` makes one per sentence, and a new line that opens on a list
 marker (`-`, `*`, `+`, `1.` or `1)`, then a space) also starts a segment, so each bullet in a set
 of notes stands on its own. Every segment starts as `unlabeled`, which lint never accepts. `init`
-refuses to overwrite a file that exists, and exits 2 on a material that does not exist or a
-`--by` it does not know, a material with nothing in it, and an `--out` folder that does not
-exist.
+refuses to overwrite a file that exists, unless `--keep` names that file (see
+[When a material changes](#when-a-material-changes)), and exits 2 on a material that does not
+exist or a `--by` it does not know, a material with nothing in it, and an `--out` folder that does
+not exist.
 
 Then name the file on the material item, as `segments:` beside `path:`, and label every segment.
 You may also move a boundary by hand, splitting one segment in two or joining two, as long as
@@ -482,9 +483,31 @@ fails test 1.
 
 The header's `sha256` pins the material as it was when it was marked. If the material changes,
 lint fails the segments file as stale (test 4), because its offsets and labels describe text that
-is no longer there. Mark it again: run `segments init` with `--out` to a new file, point the
-material item at it, and label every segment, carrying labels over from the old file wherever the
-text did not change.
+is no longer there. Mark it again, keeping the labels of every stretch of text that did not change:
+
+```bash
+npx @supersuit/hyperspec segments init materials/voice-memo.md --id voice-memo --keep materials/voice-memo.md.segments.jsonl
+```
+
+`--keep <old>` splits the material as it reads now, and every segment whose text, trimmed, a
+segment in `<old>` has keeps that segment's `id`, its `label` and every other key it carries
+(`own`, `source`, `teller`, `speaker`, anything added by hand), with its `start` and `end` taken
+from the new split. The `id` is kept because the spine cites segments by id, so a citation keeps
+pointing at the words it pointed at. Old segments with the same text are used in order, each once.
+A segment no old one matches is new or changed text: it starts `unlabeled`, gets the next `s<n>` id
+no old segment used, and is listed with the start of its text, so only those need a label.
+`--keep` may name the file being written, which is how a material is re-marked in place; with no
+`--keep`, or one naming another file, an existing file is still never overwritten. It exits 2 when
+the `--keep` file cannot be read, has a line that is not a JSON object, or marks a material other
+than `--id`.
+
+After the author's framing line, s4, is edited to "and I kept the note in my wallet.", that prints:
+
+```text
+5 segments written to materials/voice-memo.md.segments.jsonl, 4 labels carried from materials/voice-memo.md.segments.jsonl, 1 to label:
+  s6 "My first manager said this to me in my second week, and I ke..."
+Label each one (claim, story, quote, stance, question, aside, private), then run hyperspec lint on the spec.
+```
 
 The hash is over the file's bytes, so a change nobody would call an edit still counts. Converting
 line endings is the common one: a material marked with LF endings reads as stale once an editor
@@ -1146,7 +1169,10 @@ marks and any parenthetical dropped. An item that says `(from Lesson 3)` reminds
 earlier term and defines nothing.
 
 A use is a whole-word match, ignoring case, where a hyphen joins a word: "context-aware" does not
-use "context", and "skills" does not use "skill". Code is not prose: fenced blocks and inline code
+use "context", and "skills" does not use "skill". A word inside a longer defined term is not a
+use of the shorter one: with "level" and "thinking level" both defined, "thinking level" and
+"thinking levels" use only "thinking level", while "level" on its own still uses "level". The quiz
+rules read a question the same way. Code is not prose: fenced blocks and inline code
 never count as a use, so `@supersuit/superskill` does not use "supersuit". A unit's own terms
 section is not a use either. Listing a word in `knows` or `audience.knows` is a decision that the
 reader already has it, and it is the only way a unit may use a word before the unit that defines it.

package/bin/hyperspec.mjs

@@ -1,5 +1,5 @@
 #!/usr/bin/env node
-import { existsSync, statSync, writeFileSync, readFileSync, mkdirSync } from "node:fs";
+import { existsSync, statSync, writeFileSync, readFileSync, mkdirSync, realpathSync } from "node:fs";
 import { dirname, resolve, join } from "node:path";
 import { loadSpec } from "../src/load.mjs";
 import { lintSpec } from "../src/rules.mjs";
@@ -12,7 +12,7 @@ import { approve } from "../src/writer.mjs";
 import { reproduce } from "../src/reproduce.mjs";
 import { regenerate } from "../src/regenerate.mjs";
 import { compare } from "../src/compare.mjs";
-import { splitSegments } from "../src/segments.mjs";
+import { splitSegments, carrySegments } from "../src/segments.mjs";
 import { sha256 } from "../src/hash.mjs";
 import { readScope, measureFeatures, writeFeatures, scopeTemplate, GOLDENS_README } from "../src/dna.mjs";
 import { str } from "../src/placeholder.mjs";
@@ -125,16 +125,21 @@ const HELP = `hyperspec <command> [options]
                                writing, --kind with it (use --form), or a folder that does not
                                exist
 
-  segments init <material> --id <mid> [--out <file>] [--by paragraph|sentence]
+  segments init <material> --id <mid> [--out <file>] [--by paragraph|sentence] [--keep <old>]
                                split a material into candidate segments, written as JSONL to
                                <material>.segments.jsonl by default; every segment starts label:
                                unlabeled, never valid in lint; label each one by hand (claim,
                                story, quote, stance, question, aside, private), then run hyperspec
                                lint on the spec; --by sentence also starts a segment at each list
-                               item (-, *, +, 1. or 1) then a space); refuses to overwrite an
-                               existing file (exit 2); exit 2 for a missing material, a material
-                               with nothing in it, an --out folder that does not exist, or a --by
-                               outside paragraph/sentence
+                               item (-, *, +, 1. or 1) then a space); --keep re-marks an edited
+                               material: every segment whose text, trimmed, an old segment in
+                               <old> has keeps that segment's id, label and every other key, and
+                               the rest are listed to label; refuses to overwrite an existing
+                               file unless --keep names it (exit 2); exit 2 for a missing
+                               material, a material with nothing in it, an --out folder that does
+                               not exist, a --by outside paragraph/sentence, or a --keep file that
+                               cannot be read, has a line that is not a JSON object, or marks
+                               another material
 
   dna init <scope-dir> --writer W --form F --audience A --purpose P
                                write a new writer-DNA scope: scope.md (writer, form, audience,
@@ -225,7 +230,7 @@ if (cmd === "segments") {
   const sub = argv[1];
 
   if (sub === "init") {
-    const parsed = parseArgs(argv.slice(2), { valueFlags: ["--id", "--out", "--by"] });
+    const parsed = parseArgs(argv.slice(2), { valueFlags: ["--id", "--out", "--by", "--keep"] });
     if (parsed.error) { console.error(parsed.error); process.exit(2); }
     const [material] = parsed.positionals;
     if (!material) { console.error("segments init needs a material path"); process.exit(2); }
@@ -237,18 +242,55 @@ if (cmd === "segments") {
     try { materialStat = statSync(material); } catch { materialStat = null; }
     if (!materialStat || !materialStat.isFile()) { console.error(`material not found: ${material}`); process.exit(2); }
     const out = parsed.values["--out"] ?? `${material}.segments.jsonl`;
-    if (existsSync(out)) { console.error(`refusing to overwrite ${out}`); process.exit(2); }
+    const keep = parsed.values["--keep"];
+    // --keep may name the file being written: re-marking in place is the point. Nothing else is
+    // ever overwritten.
+    const samePath = (a, b) => { try { return realpathSync(a) === realpathSync(b); } catch { return resolve(a) === resolve(b); } };
+    if (existsSync(out) && !(keep && samePath(keep, out))) { console.error(`refusing to overwrite ${out}${keep ? " (--keep names a different file)" : ""}`); process.exit(2); }
     const outFolder = dirname(resolve(out));
     if (!existsSync(outFolder) || !statSync(outFolder).isDirectory()) { console.error(`folder does not exist: ${dirname(out)}; create it first`); process.exit(2); }
 
+    // The old segments to carry labels from: every line after the header, each a JSON object.
+    let old = null;
+    if (keep) {
+      let raw;
+      try { raw = readFileSync(keep, "utf8"); } catch { console.error(`--keep file not found: ${keep}`); process.exit(2); }
+      const rows = raw.split("\n").map((l, i) => ({ n: i + 1, l })).filter((r) => r.l.trim());
+      old = [];
+      for (const { n, l } of rows) {
+        let o;
+        try { o = JSON.parse(l); } catch { o = null; }
+        if (!o || typeof o !== "object" || Array.isArray(o)) { console.error(`--keep file ${keep} line ${n} is not a JSON object; fix it, or mark the material from scratch`); process.exit(2); }
+        if (n === 1) {
+          if (str(o.material) && str(o.material) !== id) { console.error(`--keep file ${keep} marks material "${o.material}", not "${id}"`); process.exit(2); }
+          continue;
+        }
+        old.push(o);
+      }
+    }
+
     const buf = readFileSync(material);
     const text = buf.toString("utf8");
     if (!/\S/.test(text)) { console.error(`nothing to mark: ${material} has no text`); process.exit(2); }
-    const segments = splitSegments(text, { by });
+    const fresh = splitSegments(text, { by });
+    const kept = old ? carrySegments(fresh, old) : null;
+    const segments = kept ? kept.segments : fresh;
     const header = { material: id, path: material, sha256: sha256(buf) };
     const lines = [JSON.stringify(header), ...segments.map((s) => JSON.stringify(s))];
     writeFileSync(out, `${lines.join("\n")}\n`);
-    console.log(`${segments.length} segments written to ${out}. Label every segment (claim, story, quote, stance, question, aside, private), then run hyperspec lint on the spec.`);
+    if (!kept) {
+      console.log(`${segments.length} segments written to ${out}. Label every segment (claim, story, quote, stance, question, aside, private), then run hyperspec lint on the spec.`);
+      process.exit(0);
+    }
+    const todo = kept.unlabeled;
+    console.log(`${segments.length} segments written to ${out}, ${kept.carried} labels carried from ${keep}, ${todo.length} to label${todo.length ? ":" : "."}`);
+    for (const s of todo) {
+      const t = s.text.replace(/\s+/g, " ").trim();
+      console.log(`  ${s.id} ${JSON.stringify(t.length > 60 ? `${t.slice(0, 60)}...` : t)}`);
+    }
+    console.log(todo.length
+      ? "Label each one (claim, story, quote, stance, question, aside, private), then run hyperspec lint on the spec."
+      : "Run hyperspec lint on the spec.");
     process.exit(0);
   }
 

package/package.json

@@ -1,6 +1,6 @@
 {
   "name": "@supersuit/hyperspec",
-  "version": "0.9.0",
+  "version": "0.9.1",
   "description": "A hyperspec is a spec written for an agent: every decision accounted for, every requirement failable and checked, every field traced. The standard and its linter.",
   "type": "module",
   "bin": {

package/src/segments.mjs

@@ -182,6 +182,50 @@ export function splitSegments(text, { by = "paragraph" } = {}) {
   return spans.map((s, i) => ({ id: `s${i + 1}`, start: s.start, end: s.end, label: "unlabeled", text: text.slice(s.start, s.end) }));
 }
 
+// -------------------------------------------------------------------------------------------
+// carrySegments: re-marking an edited material without relabeling what did not change (0.9.1).
+// A label is a fact about a stretch of text, so while the same text (compared trimmed) is still in
+// the material, its label is still true. `fresh` is what splitSegments returned for the material as
+// it reads now; `old` is the segment objects of the file it was marked in before. Each fresh
+// segment whose trimmed text an old segment has takes that old segment's id, label and every other
+// key it carries (own, source, teller, speaker, anything a person added), with start, end and text
+// from the new split. The id is carried too, because the spine cites segments by id: "m1#s3" keeps
+// pointing at the words it pointed at. Old segments with the same text are used in document order,
+// each once. A fresh segment no old one matches stays "unlabeled" and gets the next id no old
+// segment used, "s<n>" counting up past the highest. Returns { segments, carried, unlabeled }:
+// carried counts the segments that took a label other than "unlabeled"; unlabeled lists every
+// segment still to label, in document order.
+export function carrySegments(fresh, old) {
+  const byText = new Map();
+  for (const o of old) {
+    if (!o || typeof o !== "object" || typeof o.text !== "string") continue;
+    const k = o.text.trim();
+    if (!byText.has(k)) byText.set(k, []);
+    byText.get(k).push(o);
+  }
+  const taken = new Set(old.map((o) => str(o?.id)).filter(Boolean));
+  let n = Math.max(0, ...[...taken].map((id) => Number(/^s(\d+)$/.exec(id)?.[1] ?? 0)));
+  const nextId = () => {
+    do n += 1; while (taken.has(`s${n}`));
+    taken.add(`s${n}`);
+    return `s${n}`;
+  };
+  const segments = [];
+  let carried = 0;
+  for (const seg of fresh) {
+    const match = byText.get(seg.text.trim())?.shift();
+    if (!match) {
+      segments.push({ id: nextId(), start: seg.start, end: seg.end, label: "unlabeled", text: seg.text });
+      continue;
+    }
+    const { id, start, end, label, text, ...extra } = match;
+    const kept = { id: str(id) || nextId(), start: seg.start, end: seg.end, label: typeof label === "string" && label ? label : "unlabeled", ...extra, text: seg.text };
+    if (kept.label !== "unlabeled") carried += 1;
+    segments.push(kept);
+  }
+  return { segments, carried, unlabeled: segments.filter((s) => s.label === "unlabeled") };
+}
+
 // -------------------------------------------------------------------------------------------
 // readSegments: parse + validate a marked-up segments file. Never throws on a bad or missing
 // file; every failure mode becomes a finding in the lint shape (test, id, severity, message, fix),

package/src/stations/sequence.mjs

@@ -27,7 +27,11 @@
 //              way used, so the rules and the question shape are that checker's.
 //
 // A use is a whole-word, case-insensitive match, where a hyphen is part of a word: "context-aware"
-// does not use "context", and "skills" does not use "skill". The station reads the outline from
+// does not use "context", and "skills" does not use "skill". A word inside a longer defined term is
+// not a use of the shorter one (hyperspec 0.9.1): with "level" and "thinking level" both defined,
+// "thinking level" and "thinking levels" use only "thinking level", and "level" on its own still
+// uses "level". Every guard that looks for a use (order, quiz order, quiz coverage) masks the
+// longer terms first. The station reads the outline from
 // disk, resolved against the spec's folder, the way claims reads its ledger. It never reads a
 // part's text for meaning: a term used in a sense other than its definition still counts as a use.
 
@@ -202,6 +206,25 @@ function firstUse(unit, prose, term) {
 // Whether `text` (code already masked) uses `term`, by the same whole-word rule as firstUse.
 const usesTerm = (text, term) => new RegExp(`(^|[^a-z0-9-])${escapeRe(term)}([^a-z0-9-]|$)`, "i").test(text);
 
+// For each defined term, the other defined terms that contain it as a whole-word phrase, longest
+// first: { level: ["thinking level"] }. A term no other term contains maps to [].
+export function longerTerms(terms) {
+  const all = [...terms];
+  return new Map(all.map((t) => [t, all.filter((u) => u !== t && usesTerm(u, t)).sort((a, b) => b.length - a.length)]));
+}
+
+// `text` with every whole-word occurrence of each `longer` term, a trailing plural "s" included,
+// blanked to spaces. Line breaks survive, so a line number still maps to the same line, and a term
+// broken across two lines is blanked on both.
+export function maskLonger(text, longer) {
+  let out = text;
+  for (const t of longer) {
+    const re = new RegExp(`(?<![a-z0-9-])${escapeRe(t).replace(/ /g, "\\s+")}s?(?![a-z0-9-])`, "gi");
+    out = out.replace(re, (m) => m.replace(/[^\n]/g, " "));
+  }
+  return out;
+}
+
 // Every quiz in the draft, read the way the first book written to this shape wrote it:
 //
 //   ## Check yourself: Part 1
@@ -308,12 +331,17 @@ export function run(spec, draft) {
   }
 
   // order: no unit before the defining one uses the term.
+  // A longer defined term that contains the term is masked first: "thinking level" is not a use of
+  // "level".
   const prose = new Map(units.map((u) => [u, proseOf(u, seq, { withTerms: false })]));
+  const longer = longerTerms(definedAt.keys());
   for (const [t, at] of definedAt) {
     if (seq.knows.has(t)) continue;
+    const sups = longer.get(t);
     for (const u of units) {
       if (u === at) break;
-      const line = firstUse(u, prose.get(u), t);
+      const lines = sups.length ? maskLonger(prose.get(u).join("\n"), sups).split("\n") : prose.get(u);
+      const line = firstUse(u, lines, t);
       if (line) findings.push(finding({ id: "station-sequence-used-before-defined", severity: "fail", message: `${U} ${u.n} uses "${t}" before ${U} ${at.n} defines it`, fix: `Rewrite ${U} ${u.n} without "${t}", define it earlier, or add it to writing.form.sequence.knows if the reader already has the word.`, line: line }));
     }
   }
@@ -362,6 +390,12 @@ function quizFindings(draft, seq, definedAt) {
   const U = seq.unit;
   const out = [];
   const { quizzes, questions, untagged } = parseQuizzes(draft, seq);
+  // Every question's text with each term's longer defined terms masked, as the order guard reads
+  // prose: "thinking level" in a question neither uses nor tests "level".
+  const longer = longerTerms(definedAt.keys());
+  const masked = (q, t) => (longer.get(t).length ? maskLonger(q.text, longer.get(t)) : q.text);
+  const uses = (q, t) => usesTerm(masked(q, t), t);
+  const tests = (q, t) => { const text = masked(q, t); return usesTerm(text, t) || usesTerm(text, `${t}s`); };
   if (!quizzes) {
     return [finding({ id: "station-sequence-quiz-missing", severity: "fail", message: `no "${seq.quiz}" heading in the draft, and writing.form.sequence.quiz names one`, fix: `Add a "## ${seq.quiz}" section of questions, or delete quiz: from the spec.` })];
   }
@@ -375,12 +409,12 @@ function quizFindings(draft, seq, definedAt) {
     }
     for (const [t, at] of definedAt) {
       if (seq.knows.has(t) || at.n <= q.unit) continue;
-      if (usesTerm(q.text, t)) out.push(finding({ id: "station-sequence-quiz-used-before-defined", severity: "fail", message: `question ${q.n}, tagged ${U} ${q.unit}, uses "${t}", which ${U} ${at.n} defines later`, fix: `Rewrite the question without "${t}", or tag it with ${U} ${at.n} or later.`, line: q.line }));
+      if (uses(q, t)) out.push(finding({ id: "station-sequence-quiz-used-before-defined", severity: "fail", message: `question ${q.n}, tagged ${U} ${q.unit}, uses "${t}", which ${U} ${at.n} defines later`, fix: `Rewrite the question without "${t}", or tag it with ${U} ${at.n} or later.`, line: q.line }));
     }
   }
   // coverage: a simple plural counts, so "skills" tests "skill".
   for (const [t, at] of definedAt) {
-    if (!questions.some((q) => usesTerm(q.text, t) || usesTerm(q.text, `${t}s`))) {
+    if (!questions.some((q) => tests(q, t))) {
       out.push(finding({ id: "station-sequence-quiz-untested", severity: "fail", message: `no quiz question tests "${t}", which ${U} ${at.n} defines`, fix: `Add a question that uses "${t}", tagged ${U} ${at.n} or later.`, line: at.line }));
     }
   }