@supersuit/hyperspec 0.3.0 → 0.4.0

package/CHANGELOG.md

@@ -1,5 +1,64 @@
 # Changelog
 
+## 0.4.0 (2026-09-29)
+
+Marking materials. Before a writing spec can pass, every material it draws on (a brain dump, a
+transcript, a set of interview notes) is split into segments, and each segment is labeled with
+what a draft may use it as: a claim with its source, the author's own claim, a story with its
+teller, a quote with its speaker, a stance, an open question, an aside, or something private.
+The spine then cites segments rather than whole files, so every claim points at the exact words
+behind it. hyperspec splits and checks; an agent or a person labels. It still calls no model.
+
+**Behavior change for 0.3 writing specs:** a writing spec's materials must now be marked. A
+material item with no `segments:` field fails test 1, so a writing spec that passed 0.3.0 fails
+until each of its materials has a segments file, written with `hyperspec segments init` and
+labeled. Specs with no profile are unaffected.
+
+- `hyperspec segments init <material> --id <mid> [--out <file>] [--by paragraph|sentence]`
+  writes `<material>.segments.jsonl`: a header naming the material, its path and the SHA-256 of
+  its bytes, then one line per segment with character offsets, the verbatim text, and the label
+  `unlabeled`. Paragraph mode is the default. It refuses to overwrite a file, and exits 2 with a
+  plain message on a missing material, a material with nothing in it, an `--out` folder that
+  does not exist, or an unknown `--by`.
+- In sentence mode, a new line that opens on a list marker (`-`, `*`, `+`, or a number followed
+  by `.` or `)`, then a space) starts a new segment, and a numbered item's own `1.` is not read
+  as a sentence ending. A bullet that is entirely a quotation ending in `."` used to run into the
+  next bullet; it now stands on its own. Paragraph mode is unchanged.
+- Seven labels, a closed set: `claim` (needs `source`, or `own: true`), `story` (needs
+  `teller`), `quote` (needs `speaker`), `stance`, `question`, `aside` and `private`. `unlabeled`
+  is never accepted. A placeholder word (`TODO`, `n/a`, `tbd`, `...`, `???` and the rest) counts
+  as missing in these fields, in the header, and in segment ids, as it does everywhere else in
+  the linter.
+- What `lint` checks on each segments file, under test 1: the file exists and parses, its header
+  names the right material, every label is from the set, ids are unique, and segments never
+  overlap and cover every character that is not whitespace (the finding quotes the first
+  uncovered text and gives its offset), and the material has some text to mark. Under test 4: every segment's text
+  matches the material word for word, each label carries the field it needs, and the material
+  has not changed since it was marked (its SHA-256 still matches). A changed material fails as
+  stale until it is marked again.
+- A spine claim may cite `material#segment`. The segment must exist (test 4), and citing a
+  `private` or `question` segment fails test 5. A bare material id still cites the whole
+  material; `material#` with nothing after the `#` fails as an unknown segment.
+- Every marking finding id starts `writing-materials-` or `writing-spine-`, and every message
+  names the material, and the segment where there is one. Paths in messages read as the spec
+  wrote them, never resolved to a folder on your machine, so `--json` output is the same
+  everywhere. WRITING.md lists every finding with its test.
+- A new export, `@supersuit/hyperspec/writing`, gives your own tools `MATERIAL_LABELS` and
+  `readSegments`, the same parse-and-check lint runs. `readSegments` returns
+  `{ header, segments, findings }` and never throws; `displayPath` and `materialDisplayPath` set
+  how the files are named in its messages.
+- `hyperspec init --profile writing` names a segments file for its placeholder material, and the
+  materials check reads "every segment of every material carries a label from the closed set,
+  matches its source verbatim, and the markings are current".
+- Both writing examples ship with every material marked. Between them they use all seven labels,
+  and every spine claim cites segments.
+- WRITING.md gains a Marking materials section: the file format with a worked sample, the labels
+  and what each needs, coverage, staleness (including a line-ending conversion, which changes the
+  hash), citing segments, every finding with its test, and the import. README and SPEC.md point
+  at it.
+- The repository's `.gitattributes` keeps example materials and test fixtures LF on every
+  checkout, so the hashes their segments files pin still match on a Windows clone.
+
 ## 0.3.0 (2026-09-29)
 
 The writing profile. A piece of writing can now carry a hyperspec that names everything an

package/README.md

@@ -25,13 +25,14 @@ improvement ledger. Every test is defined in [SPEC.md](SPEC.md).
 | `hyperspec lint <file...> [--json]` | Score each hyperspec against the nine tests. |
 | `hyperspec init <file> [--title T] [--kind K]` | Write a new hyperspec skeleton. Refuses to overwrite an existing file. |
 | `hyperspec init <file> --profile writing [--title T] [--form F] [--fiction]` | Write a writing-spec skeleton, every block shown with placeholders. |
+| `hyperspec segments init <material> --id <mid> [--out F] [--by paragraph\|sentence]` | Split a material into segments to label. Refuses to overwrite an existing file. |
 | `hyperspec recipe check <output-or-recipe>` | Check that a recipe records everything the standard asks for. |
 | `hyperspec recipe approve <recipe> --by <slug>` | Record who approved the output. |
 | `hyperspec reproduce <recipe> [--restore]` | Re-check every hash the recipe recorded. Never runs a model. |
 | `hyperspec regenerate <recipe> --out <path> --clicker <slug> <one change> [--run cmd]` | Make a child recipe from a parent and one named change, rerunning only the stages it reaches. |
 | `hyperspec compare <child-recipe> --doctor cmd` | Grade a child and its parent through one doctor against one spec. |
 
-Every command except `init` takes `--json`. `hyperspec --help` prints every flag.
+Every command except `init` and `segments init` takes `--json`. `hyperspec --help` prints every flag.
 
 ## Exit codes
 
@@ -131,6 +132,26 @@ each rule reports under are in [WRITING.md](WRITING.md). Two complete specs that
 nothing to warn ship in `examples/writing/`: an essay for new managers, and a short story with
 two characters whose voices a judge can tell apart.
 
+### Marking materials
+
+Every material a writing spec draws on is marked before the spec can pass: split into segments,
+and each segment labeled with what it may be used as (claim, story, quote, stance, question,
+aside, private). hyperspec does the splitting and the checking; an agent or a person does the
+labeling.
+
+```bash
+npx @supersuit/hyperspec segments init materials/voice-memo.md --id voice-memo
+npx @supersuit/hyperspec lint essay.hyperspec.md
+```
+
+The first writes `materials/voice-memo.md.segments.jsonl`, every segment `unlabeled`. Name that
+file as `segments:` on the material item, label every segment, and `lint` checks that each label
+is from the set and carries what it needs, that each segment matches the material word for word,
+that the material has not changed since, and that no spine claim cites a private or question
+segment. The file format, the labels, and every finding are in
+[WRITING.md](WRITING.md#marking-materials). A tool of your own can run the same check with
+`import { readSegments } from "@supersuit/hyperspec/writing"`.
+
 ## The format
 
 A hyperspec is a markdown file with a YAML frontmatter block: `decisions`, `requirements`,

package/SPEC.md

@@ -97,7 +97,7 @@ examples:
   - path: examples/minimal.hyperspec.md
     why: the smallest spec that passes all nine tests
 resume:
-  next_action: collect adopter issues on 0.3, the writing profile included, and cut 0.4 from them
+  next_action: collect adopter issues on 0.4, materials marking included, and cut 0.5 from them
 feedback:
   issues: https://github.com/SupersuitUp/hyperspec/issues
   fork: MIT; fork it for your own purposes and say so in your SPEC
@@ -109,7 +109,7 @@ improvement:
 
 A person writing for another person leaves most of the specification unsaid, because the other person fills the gaps from shared context. An agent has none of that context, so it fills every gap with the average, and the average is what reads as middling. Hyperspecification is writing down the gaps. It is a level of detail that would feel like overkill between two people and is exactly enough for an agent: every decision the agent would otherwise guess is either decided, delegated with the rule for deciding it, or marked open, so the work stops instead of guessing.
 
-**Version 0.3.0** (2026-09-29)
+**Version 0.4.0** (2026-09-29)
 
 ## What makes a spec a hyperspec
 
@@ -220,6 +220,8 @@ Each row lists every condition under which `hyperspec lint` fails that test. A w
 | 8 its adopters can push back on it | `feedback.issues` or `feedback.fork` missing |
 | 9 it improves itself | `improvement.ledger` missing; a ledger path that exists and is not a readable file; if the ledger file exists, a line that is not a JSON object, a `verdict` outside one-shot, improved or not-improved, `improved` without `change`, `not-improved` without `reason`. A declared ledger that does not exist yet is a warning |
 
+A profile adds its own conditions to these rows. The writing profile's are in [WRITING.md](WRITING.md#the-test-mapping), including the checks on each material's segments file: every material marked, every segment labeled from the closed set and matching its material word for word, the marking current (tests 1 and 4), and no spine claim citing a private or question segment (test 5).
+
 ## Exit codes
 
 `hyperspec lint` reports the worst result across every file it is given:
@@ -243,7 +245,7 @@ Silence is not a verdict. A run that learned nothing has to say so and why, and
 
 A profile adds the rules for one kind of work on top of the nine tests. A spec opts in with a top-level `profile:` naming it. A profile never adds a tenth test: every finding it raises reports under one of the nine, with an id that starts with the profile's name, and the score stays out of nine. `lint` prints one more line for a profiled spec, how many of the profile's blocks are complete. A `profile` this linter does not know is a warning under test 7, and none of its rules are checked.
 
-One profile ships: `writing`, for essays, chapters, letters, stories and anything else an agent drafts for a person to read. Its blocks, its fields, which test each rule reports under, and `hyperspec init --profile writing` are in [WRITING.md](WRITING.md).
+One profile ships: `writing`, for essays, chapters, letters, stories and anything else an agent drafts for a person to read. Its blocks, its fields, which test each rule reports under, `hyperspec init --profile writing`, and marking materials with `hyperspec segments init` are in [WRITING.md](WRITING.md).
 
 ## Recipes
 

package/WRITING.md

@@ -46,7 +46,7 @@ thinking out loud, `considered` for something someone has reviewed, `verified` f
 checked against its source. A brain dump is the most valuable input and the least structured, so
 marking it is the step that turns thinking into something an agent can cite. Each material is
 split into segments and each segment gets a label saying what it may be used as (see
-[Materials labels](#materials-labels)).
+[Marking materials](#marking-materials)).
 
 ### 2. Writer DNA: who is writing, and how they sound, for this purpose
 
@@ -104,8 +104,8 @@ chapters around it, a wiki article checks its links, a text message checks bubbl
 ### 7. Spine: what it argues
 
 The kind of argument (thesis, testimony, primer, letter, story; the set is open) and the claim
-chain: the three to seven claims the piece has to land, in order, each pointing at the materials
-that support it.
+chain: the three to seven claims the piece has to land, in order, each pointing at the marked
+segments that support it.
 
 ### 8. Sources and claims: what lets another agent pick it up
 
@@ -159,12 +159,13 @@ writing:
     items:                           # at least one
       - id: voice-memo               # unique across items
         path: materials/voice-memo.md
+        segments: materials/voice-memo.md.segments.jsonl   # written by hyperspec segments init, then labeled
         produced_by: example-author
         captured: "2026-09-12"
         how: voice memo, transcribed
         trust: raw                   # raw | considered | verified
     check:
-      station: every segment of every material carries a label from the closed set
+      station: every segment of every material carries a label from the closed set, matches its source verbatim, and the markings are current
     source: capture step
     author: agent:claude
   dna:
@@ -246,16 +247,17 @@ writing:
     claims:                          # 3 to 7, in order, each with its own id
       - id: c1
         text: the first one-on-one is the one meeting the report should set the agenda for
-        materials:                   # ids from materials.items; voice-memo#segment is accepted
-          - voice-memo
+        materials:                   # material#segment cites one segment; never a private or question one
+          - voice-memo#s3
       - id: c2
         text: status belongs in the tracker
-        materials:
+        materials:                   # a bare material id cites the whole material
           - voice-memo
       - id: c3
         text: three questions are enough to hand the meeting over
         materials:
-          - voice-memo
+          - voice-memo#s2
+          - voice-memo#s5
     check:
       rubric: each claim lands, in order, and nothing is argued outside the chain
     source: spine interview
@@ -317,11 +319,11 @@ Each row lists what the writing profile adds to that test. The core conditions i
 
 | Test | A writing spec fails it when |
 |---|---|
-| 1 every decision is accounted for | a required block is missing and not deferred; a required field is missing; a closed-set value is outside its set (`trust`, `reader`, `change.kind`, the shape of `identity`, `unsourced_claim`); `identity: character:<id>` names a character that is not in `writing.characters`; `fiction` is present and is anything other than `true` or `false`; two materials, two spine claims or two characters share an id; `form.length.min` or `max` is not a whole number of at least 1, or `min` is greater than `max`; `spine.claims` has fewer than three or more than seven distinct claims; a character has no knowledge entry, or an entry lacks `by` or `knows`. A `stance` outside the four is a warning, and so is `unsourced_claim: warn` |
+| 1 every decision is accounted for | a required block is missing and not deferred; a required field is missing; a closed-set value is outside its set (`trust`, `reader`, `change.kind`, the shape of `identity`, `unsourced_claim`); `identity: character:<id>` names a character that is not in `writing.characters`; `fiction` is present and is anything other than `true` or `false`; two materials, two spine claims or two characters share an id; `form.length.min` or `max` is not a whole number of at least 1, or `min` is greater than `max`; `spine.claims` has fewer than three or more than seven distinct claims; a character has no knowledge entry, or an entry lacks `by` or `knows`; a material has no text, or no `segments` field, or its segments file is missing, malformed, labels a segment outside the seven (`unlabeled` included), repeats a segment id, or has segments that overlap or leave text uncovered. A `stance` outside the four is a warning, and so is `unsourced_claim: warn` |
 | 2 every requirement can fail | `goal.conditions` lists fewer than five or more than ten distinct ids, lists an id twice, or names an id that is not a top-level requirement |
 | 3 every requirement names its check | a block or a character has no `check` with a `station` or a `rubric` |
-| 4 every field says where it came from and who wrote it | a block or a character has no `source` or no `author`; a spine claim names no materials, or names a material id that is not in `materials.items` |
-| 5 negative space is specified | `persona.will_not_say` is empty; `persona.facts_from` is anything other than `sources` |
+| 4 every field says where it came from and who wrote it | a block or a character has no `source` or no `author`; a spine claim names no materials, or names a material id that is not in `materials.items`, or a segment that is not in that material's segments file; a segment's text does not match its material word for word; a material changed after it was marked; a claim segment has no `source` and no `own`, a story no `teller`, a quote no `speaker` |
+| 5 negative space is specified | `persona.will_not_say` is empty; `persona.facts_from` is anything other than `sources`; a spine claim cites a `private` or a `question` segment |
 | 6 examples outrank adjectives | a golden has no `why`; a material, `dna.rules`, golden or character `entity` path does not exist or is not a file; a character has no golden lines or no rejected lines, or has the same line in both (compared trimmed and case-folded) |
 | 7 a stranger can resume it | `writing.progress` exists. An unknown `profile:` is a warning |
 | 8 its adopters can push back on it | nothing further; the core rule applies |
@@ -342,25 +344,188 @@ Each row lists what the writing profile adds to that test. The core conditions i
 
 `form.name` and `spine.kind` are open: name the form and the kind of argument in your own words.
 
-## Materials labels
+## Marking materials
+
+A brain dump mixes things a draft may use with things it may not: a checked fact, an opinion, a
+story from the author's own week, a line said in confidence. Marking tells them apart before an
+agent drafts anything. Each material is split into segments, each segment gets one label saying
+what it may be used as, and the spine cites segments, so every claim in the piece points at the
+exact words that support it.
+
+hyperspec never decides a label. `segments init` splits a material the same way every time, an
+agent or a person labels each segment by editing the file it wrote, and `lint` checks everything
+a rule can check: every segment carries a label from the closed set and the field that label
+needs, matches its material word for word, and was marked against the material as it reads now.
+
+A material item with no `segments` field fails test 1. Marking comes before specifying, so a
+writing spec cannot pass until every material it draws on is marked.
+
+### Marking a material
+
+```bash
+npx @supersuit/hyperspec segments init materials/voice-memo.md --id voice-memo
+```
+
+`hyperspec segments init <material> --id <mid> [--out <file>] [--by paragraph|sentence]` writes
+`<material>.segments.jsonl`, or the path `--out` names. `--by paragraph`, the default, makes one
+segment per paragraph. `--by sentence` makes one per sentence, and a new line that opens on a list
+marker (`-`, `*`, `+`, `1.` or `1)`, then a space) also starts a segment, so each bullet in a set
+of notes stands on its own. Every segment starts as `unlabeled`, which lint never accepts. `init`
+refuses to overwrite a file that exists, and exits 2 on a material that does not exist or a
+`--by` it does not know, a material with nothing in it, and an `--out` folder that does not
+exist.
+
+Then name the file on the material item, as `segments:` beside `path:`, and label every segment.
+You may also move a boundary by hand, splitting one segment in two or joining two, as long as
+the rules under [Coverage](#coverage) still hold.
+
+### The segments file
 
-Before a material is used, it is split into segments, and each segment gets one of seven labels.
-The label decides what the segment may become in the draft.
+JSON Lines: one object per line. Line 1 is a header, and every later line is one segment. For this
+material, `materials/voice-memo.md`:
 
-| Label | Means | May be used as |
+```text
+Voice memo, recorded on a walk. Raw thinking.
+
+My first one-on-one as a manager was a disaster. I ran it from my own list.
+
+The first one-on-one is the one meeting the report should set the agenda for.
+
+My first manager said this to me in my second week, and I wrote it down.
+
+"Ask what they want to talk about, then stop talking."
+```
+
+`segments init` writes five segments, and once they are labeled the file reads:
+
+```jsonl
+{"material":"voice-memo","path":"materials/voice-memo.md","sha256":"66c62b995a6f29c72f2a9a20c6b27deaef5d9b195bb6bab96336c06f9a8f2bfc"}
+{"id":"s1","start":0,"end":45,"label":"aside","text":"Voice memo, recorded on a walk. Raw thinking."}
+{"id":"s2","start":47,"end":122,"label":"story","teller":"example-author","text":"My first one-on-one as a manager was a disaster. I ran it from my own list."}
+{"id":"s3","start":124,"end":201,"label":"claim","own":true,"text":"The first one-on-one is the one meeting the report should set the agenda for."}
+{"id":"s4","start":203,"end":275,"label":"story","teller":"example-author","text":"My first manager said this to me in my second week, and I wrote it down."}
+{"id":"s5","start":277,"end":331,"label":"quote","speaker":"the author's first manager","text":"\"Ask what they want to talk about, then stop talking.\""}
+```
+
+The author's framing, s4, is a segment of its own, so the quote, s5, holds only the manager's
+words, which is all a `quote` may hold.
+
+- **Header.** `material` is the item's id and must match it. `path` records the material path
+  given to `segments init`; lint reads the material from the item's own `path`. `sha256` is the
+  SHA-256 of the material file's bytes when it was marked.
+- **`id`** is unique within the file. `init` writes `s1`, `s2` and so on; any id works, and it is
+  what the spine cites.
+- **`start` and `end`** are character offsets into the material's text read as UTF-8, counted as
+  JavaScript string indices (UTF-16 code units), with `end` exclusive.
+- **`text`** is exactly the material's characters from `start` to `end`.
+- **`label`**, plus the one field some labels need (below). Those fields are strings, except `own`.
+
+### The labels
+
+| Label | Means | May be used as | Needs |
+|---|---|---|---|
+| `claim` | a statement of fact about the world | only with a source, or as the author's own claim said as such | `source`, non-empty, or `own: true` |
+| `story` | something that happened, told by someone who was there | testimony, with the teller named | `teller` |
+| `quote` | words someone said, verbatim | quoted exactly, never paraphrased inside quotation marks | `speaker` |
+| `stance` | an opinion or conviction | the author's position | nothing more |
+| `question` | something open | a prompt for the interview, never an assertion | nothing more |
+| `aside` | true but off the thread | held back unless the spine needs it | nothing more |
+| `private` | not for this audience | never used; kept for context | nothing more |
+
+`own` counts when it is `true` or the string `"true"`. Any other value, `false` included, leaves
+it unset, and a claim with no `source` then fails. A placeholder word such as `TODO`, `n/a` or
+`???` counts as missing here as it does everywhere in a hyperspec (see [The schema](#the-schema)),
+so `source: "TODO"` fails like no source at all. The same holds for the header's fields and for
+segment ids.
+
+### Coverage
+
+Taken in order of `start`, whatever order the lines are in, segments never overlap, and between
+them they cover every character of the material that is not whitespace. Whitespace between
+segments may be left out, which is what `init` does. Segment ids are unique within a file. When
+text is left uncovered, the finding gives the offset of the first uncovered stretch and quotes up
+to 60 characters of it. A material with no text that is not whitespace has nothing to mark and
+fails test 1.
+
+### When a material changes
+
+The header's `sha256` pins the material as it was when it was marked. If the material changes,
+lint fails the segments file as stale (test 4), because its offsets and labels describe text that
+is no longer there. Mark it again: run `segments init` with `--out` to a new file, point the
+material item at it, and label every segment, carrying labels over from the old file wherever the
+text did not change.
+
+The hash is over the file's bytes, so a change nobody would call an edit still counts. Converting
+line endings is the common one: a material marked with LF endings reads as stale once an editor
+or a checkout setting rewrites it with CRLF. Mark a material in the line endings it will be kept
+in, and if it lives in git, pin them with a `.gitattributes` line such as
+`materials/** text eol=lf`.
+
+### Citing segments in the spine
+
+A spine claim cites a segment as `<material>#<segment>`, such as `voice-memo#s3`. The segment has
+to exist in that material's segments file (test 4). A `private` segment is never used and a
+`question` is never an assertion, so a claim citing either fails test 5. A bare material id, such
+as `voice-memo`, still cites the whole material; `voice-memo#`, with nothing after the `#`, is not
+a bare id and fails as an unknown segment. When a claim cites a segment of a material whose
+segments cannot be read at all (the material is not marked, its file is missing, or the file
+holds no segments), lint says so once for that material rather than once per citation.
+
+### Findings
+
+Every marking finding fails the test in its row. `<segment>` is the segment's id, or its
+position when it has none; `<line>` is a line number in the segments file; `<n>` is the claim's
+position in `spine.claims`, counting from 0. Every message names the material, and the segment
+where there is one, and prints paths as the spec wrote them, so the output is the same on every
+machine.
+
+| Id | Test | Fails when |
 |---|---|---|
-| `claim` | a statement of fact about the world | only with a source, or as the author's own claim said as such |
-| `story` | something that happened, told by someone who was there | testimony, with the teller named |
-| `quote` | words someone said, verbatim | quoted exactly, never paraphrased inside quotation marks |
-| `stance` | an opinion or conviction | the author's position |
-| `question` | something open | a prompt for the interview, never an assertion |
-| `aside` | true but off the thread | held back unless the spine needs it |
-| `private` | not for this audience | never used; kept for context |
-
-The linter defines the set once, as `MATERIAL_LABELS` in `src/writing.mjs`. This release checks
-the materials list itself; it does not yet read segment files or check their labels. A spine
-claim may point at a segment as `m1#segment`, and today only the material id before the `#` is
-checked.
+| `writing-materials-unmarked` | 1 | a material item has no `segments` field |
+| `writing-materials-segments-missing` | 1 | the segments file does not exist or cannot be read |
+| `writing-materials-material-missing` | 1 | the material file cannot be read. Lint reports a missing material path under test 6 instead, so this comes only from `readSegments` |
+| `writing-materials-empty` | 1 | the material has no text that is not whitespace |
+| `writing-materials-header` | 1 | line 1 is not a JSON object, or has no `material`, `path` or `sha256` |
+| `writing-materials-header-material` | 1 | the header names a different material from the item |
+| `writing-materials-json-line-<line>` | 1 | a segment line is not a JSON object |
+| `writing-materials-segment-id-<line>` | 1 | a segment has no id |
+| `writing-materials-segment-id` | 1 | two segments share an id |
+| `writing-materials-label-<segment>` | 1 | a label outside the seven, `unlabeled` included |
+| `writing-materials-segment-shape-<segment>` | 1 | `start` and `end` are not whole numbers with `start` at least 0, `end` greater than `start`, and `end` no further than the material's length |
+| `writing-materials-overlap` | 1 | two segments overlap |
+| `writing-materials-coverage` | 1 | text that is not whitespace lies outside every segment |
+| `writing-materials-text-<segment>` | 4 | `text` is not the material's characters from `start` to `end` |
+| `writing-materials-stale` | 4 | the material's SHA-256 no longer matches the header |
+| `writing-materials-claim-source-<segment>` | 4 | a claim has no `source` and no `own` |
+| `writing-materials-story-teller-<segment>` | 4 | a story has no `teller` |
+| `writing-materials-quote-speaker-<segment>` | 4 | a quote has no `speaker` |
+| `writing-spine-materials-segments-unresolvable-<material>` | 4 | a claim cites a segment of a material whose segments cannot be read |
+| `writing-spine-claim-<n>-materials-segment-unknown` | 4 | a claim cites a segment that is not in the file |
+| `writing-spine-claim-<n>-materials-segment-private` | 5 | a claim cites a `private` segment |
+| `writing-spine-claim-<n>-materials-segment-question` | 5 | a claim cites a `question` segment |
+
+### Reading segments from your own tool
+
+A tool that labels materials, such as an agent's capture step or an editor, can import the label
+set and the same parse-and-check lint runs:
+
+```js
+import { MATERIAL_LABELS, readSegments } from "@supersuit/hyperspec/writing";
+
+const { header, segments, findings } = readSegments("materials/voice-memo.md.segments.jsonl", {
+  materialPath: "materials/voice-memo.md",
+  materialId: "voice-memo",
+});
+```
+
+`readSegments` never throws. It returns the parsed header (or `null`), every segment line that
+parsed as a JSON object, and findings in the shape lint reports: `test`, `id`, `severity`,
+`message` and `fix`. Without `materialPath` it runs only the checks that need no material text
+(the header, ids, labels and label fields); with it, it also checks verbatim text, coverage,
+overlap and staleness. `materialId`, when given, has to match the header's `material`.
+Two more options, `displayPath` and `materialDisplayPath`, set how the two files are named in
+messages (lint passes the paths as the spec wrote them); by default the paths are printed as
+given. `MATERIAL_LABELS` is the seven labels, in the order of the table above.
 
 ## Deferring a block
 
@@ -394,7 +559,9 @@ npx @supersuit/hyperspec init story.hyperspec.md --profile writing --form "short
 The skeleton shows every required block in schema order with every field present as a `TODO`
 placeholder. `dna`, `persona`, `audience` and `goal` also carry an open decision whose question
 says what you have to answer before the placeholder means anything. `--form` sets both `kind:`
-and `writing.form.name`, and defaults to `essay`. `--fiction` sets `fiction: true` and adds one
+and `writing.form.name`, and defaults to `essay`. The material item names
+`materials/TODO.md.segments.jsonl`, the file `segments init` writes for `materials/TODO.md`, so
+materials keeps failing until a real material is marked. `--fiction` sets `fiction: true` and adds one
 character with the same treatment. The skeleton never passes: it lints `fail`, with
 `writing: 1/9 blocks complete` (or `0/9` with `--fiction`), until the placeholders and the open
 decisions are replaced with real content.
@@ -415,12 +582,16 @@ names:
   Each character has speech rules, a knowledge timeline by scene, and golden and rejected lines
   in a voice you can tell apart from the other's.
 
+Every material in both is marked. Between them the two examples use all seven labels, each with
+the field it needs, and every spine claim cites the segments that support it. Each segments file
+keeps the boundaries `segments init` wrote, in paragraph mode for prose and sentence mode for
+bulleted notes, so you can re-run it and compare.
+
 Both lint `pass (9/9)` with `writing: 9/9 blocks complete` and no findings. A test runs them on
 every release, so they cannot drift from the linter.
 
 ## What later versions add
 
-This release is the schema and its lint. Later versions build on it in order: marking materials
-(a brain dump or transcript in, labeled segments out, with the labels above enforced), scoped
-DNA with annotated goldens filed by form, audience and purpose, and the stations themselves,
-running the checks each block names and grading drafts against the goal.
+This release is the schema, its lint, and marked materials. Later versions build on it in order:
+scoped DNA with annotated goldens filed by form, audience and purpose, and the stations
+themselves, running the checks each block names and grading drafts against the goal.

package/bin/hyperspec.mjs

@@ -1,5 +1,5 @@
 #!/usr/bin/env node
-import { existsSync, statSync, writeFileSync } from "node:fs";
+import { existsSync, statSync, writeFileSync, readFileSync } from "node:fs";
 import { dirname, resolve } from "node:path";
 import { loadSpec } from "../src/load.mjs";
 import { lintSpec } from "../src/rules.mjs";
@@ -12,6 +12,8 @@ import { approve } from "../src/writer.mjs";
 import { reproduce } from "../src/reproduce.mjs";
 import { regenerate } from "../src/regenerate.mjs";
 import { compare } from "../src/compare.mjs";
+import { splitSegments } from "../src/segments.mjs";
+import { sha256 } from "../src/hash.mjs";
 
 const HELP = `hyperspec <command> [options]
 
@@ -30,6 +32,17 @@ const HELP = `hyperspec <command> [options]
                                writing, --kind with it (use --form), or a folder that does not
                                exist
 
+  segments init <material> --id <mid> [--out <file>] [--by paragraph|sentence]
+                               split a material into candidate segments, written as JSONL to
+                               <material>.segments.jsonl by default; every segment starts label:
+                               unlabeled, never valid in lint; label each one by hand (claim,
+                               story, quote, stance, question, aside, private), then run hyperspec
+                               lint on the spec; --by sentence also starts a segment at each list
+                               item (-, *, +, 1. or 1) then a space); refuses to overwrite an
+                               existing file (exit 2); exit 2 for a missing material, a material
+                               with nothing in it, an --out folder that does not exist, or a --by
+                               outside paragraph/sentence
+
   recipe check <output-or-recipe> [--json]
                                check a recipe's completeness (a path not ending .recipe.json
                                means <path>.recipe.json)
@@ -98,6 +111,41 @@ if (cmd === "init") {
   process.exit(0);
 }
 
+if (cmd === "segments") {
+  const sub = argv[1];
+
+  if (sub === "init") {
+    const parsed = parseArgs(argv.slice(2), { valueFlags: ["--id", "--out", "--by"] });
+    if (parsed.error) { console.error(parsed.error); process.exit(2); }
+    const [material] = parsed.positionals;
+    if (!material) { console.error("segments init needs a material path"); process.exit(2); }
+    if (!parsed.values["--id"]) { console.error("segments init needs --id <material id>"); process.exit(2); }
+    const id = parsed.values["--id"];
+    const by = parsed.values["--by"] ?? "paragraph";
+    if (by !== "paragraph" && by !== "sentence") { console.error(`--by must be paragraph or sentence, not "${by}"`); process.exit(2); }
+    let materialStat;
+    try { materialStat = statSync(material); } catch { materialStat = null; }
+    if (!materialStat || !materialStat.isFile()) { console.error(`material not found: ${material}`); process.exit(2); }
+    const out = parsed.values["--out"] ?? `${material}.segments.jsonl`;
+    if (existsSync(out)) { console.error(`refusing to overwrite ${out}`); process.exit(2); }
+    const outFolder = dirname(resolve(out));
+    if (!existsSync(outFolder) || !statSync(outFolder).isDirectory()) { console.error(`folder does not exist: ${dirname(out)}; create it first`); process.exit(2); }
+
+    const buf = readFileSync(material);
+    const text = buf.toString("utf8");
+    if (!/\S/.test(text)) { console.error(`nothing to mark: ${material} has no text`); process.exit(2); }
+    const segments = splitSegments(text, { by });
+    const header = { material: id, path: material, sha256: sha256(buf) };
+    const lines = [JSON.stringify(header), ...segments.map((s) => JSON.stringify(s))];
+    writeFileSync(out, `${lines.join("\n")}\n`);
+    console.log(`${segments.length} segments written to ${out}. Label every segment (claim, story, quote, stance, question, aside, private), then run hyperspec lint on the spec.`);
+    process.exit(0);
+  }
+
+  console.error(`unknown segments subcommand: ${sub}\n\n${HELP}`);
+  process.exit(2);
+}
+
 if (cmd === "lint") {
   const json = argv.includes("--json");
   // Lint takes one flag, --json, and no flag takes a value, so a flag is dropped on its own and
@@ -359,7 +407,7 @@ if (cmd === "compare") {
   if (json) console.log(JSON.stringify(result, null, 2));
 
   // Both a usage error and a check-failed error (missing output, escaping path, ...) map to
-  // 2 here — compare's own ok:false without a usage flag is still "could not grade", i.e. an
+  // 2 here: compare's own ok:false without a usage flag is still "could not grade", i.e. an
   // unreadable-input class failure, not a graded-but-worse-1 class one.
   if (!result.ok) {
     if (!json) console.error(result.error);

package/examples/writing/essay/materials/interview-notes.md.segments.jsonl

@@ -0,0 +1,10 @@
+{"material":"interview","path":"essay/materials/interview-notes.md","sha256":"3c4fd3ff19435027b1bb9584e648128d59feb3184f67eeaf6b09ed9fa8bcfaeb"}
+{"id":"s1","start":0,"end":118,"label":"aside","text":"Interview notes, a thirty-minute call with an engineering manager of eight years, taken by the\nauthor during the call."}
+{"id":"s2","start":119,"end":199,"label":"aside","text":"Considered: the manager reviewed these notes afterwards and corrected two\nlines."}
+{"id":"s3","start":201,"end":240,"label":"claim","source":"interview with an engineering manager of eight years","text":"- Her rule: the report owns the agenda."}
+{"id":"s4","start":241,"end":356,"label":"claim","source":"interview with an engineering manager of eight years","text":"She keeps a shared document per person; they add items\n  before the meeting, and she adds hers last, at the bottom."}
+{"id":"s5","start":357,"end":448,"label":"quote","speaker":"the engineering manager interviewed","text":"- \"If I have something urgent, it is not a one-on-one topic. I send it the day it happens.\""}
+{"id":"s6","start":449,"end":617,"label":"claim","source":"interview with an engineering manager of eight years","text":"- The first one-on-one with a new report is always the same: she asks how they like to receive\n  feedback, in writing or out loud, right away or at the end of the week."}
+{"id":"s7","start":618,"end":673,"label":"claim","source":"interview with an engineering manager of eight years","text":"- Her warning: new managers treat silence as a problem."}
+{"id":"s8","start":674,"end":733,"label":"quote","speaker":"the engineering manager interviewed","text":"\"Wait. Count to five. The real answer is\n  the second one.\""}
+{"id":"s9","start":734,"end":812,"label":"claim","source":"interview with an engineering manager of eight years","text":"- She cancels a one-on-one only when the report asks her to, never on her own."}

package/examples/writing/essay/materials/team-survey.md.segments.jsonl

@@ -0,0 +1,6 @@
+{"material":"survey","path":"essay/materials/team-survey.md","sha256":"ae2c821b1731ad02eb3280c28872aa9d2163728b868f262554d0cbfd3da19b97"}
+{"id":"s1","start":0,"end":139,"label":"aside","text":"Summary of an internal survey the author's team ran in the spring, 41 responses, figures checked\nagainst the raw export by a second person."}
+{"id":"s2","start":140,"end":149,"label":"aside","text":"Verified."}
+{"id":"s3","start":151,"end":261,"label":"claim","source":"team survey, spring, 41 responses, figures checked against the raw export","text":"- 29 of 41 said their most useful one-on-one in the last quarter was one where they brought the\n  first topic."}
+{"id":"s4","start":262,"end":358,"label":"claim","source":"team survey, spring, 41 responses, figures checked against the raw export","text":"- 11 of 41 said at least one of their one-on-ones in the last quarter was mostly project status."}
+{"id":"s5","start":359,"end":449,"label":"claim","source":"team survey, spring, 41 responses, figures checked against the raw export","text":"- The most common free-text request, in 9 responses: \"ask me what I want to work on next.\""}

package/examples/writing/essay/materials/voice-memo.md.segments.jsonl

@@ -0,0 +1,7 @@
+{"material":"voice-memo","path":"essay/materials/voice-memo.md","sha256":"c1e07e7d6886f0f3abe638500b614a94883b66f259d272b213f7fe63e70afdc8"}
+{"id":"s1","start":0,"end":87,"label":"aside","text":"Voice memo transcript, recorded by the author on a walk, lightly cleaned. Raw thinking."}
+{"id":"s2","start":89,"end":385,"label":"story","teller":"example-author","text":"My first one-on-one as a manager was a disaster, and the reason was simple: I ran it. I had a\nlist. I went down the list. Project status, blockers, the thing from Tuesday. Thirty minutes\nlater my report said \"cool, thanks\" and left, and I had learned nothing I could not have read in\nthe tracker."}
+{"id":"s3","start":387,"end":572,"label":"claim","own":true,"text":"What I wish someone had told me: the first one-on-one is the only meeting where they get to set\nthe agenda. If you set it, you have told them what the meeting is for, and it is for you."}
+{"id":"s4","start":574,"end":799,"label":"claim","own":true,"text":"The three questions I use now. What is taking more of your energy than it should? What do you\nwant to be doing more of in six months? What should I stop doing, or start doing, that would\nmake your week easier? Then I shut up."}
+{"id":"s5","start":801,"end":897,"label":"stance","text":"Status goes in the tracker. If a one-on-one is a status meeting, cancel it and read the tracker."}
+{"id":"s6","start":899,"end":1037,"label":"aside","text":"Aside, probably not for this piece: my second manager used to walk the one-on-ones outside. I\nliked it but I do not think it is the point."}