@eriknaslund/skiss 0.5.0 → 0.6.1

package/README.md

@@ -43,6 +43,7 @@ Skiss is text you can type while someone talks. A field is one line. A change is
 - **`~` and `=` are two different statements.** `~` says two classes are about the same thing. `=` says which field joins to which. The first is a domain conversation, the second an integration conversation.
 - **`<` inherits, once.** `Jedi < Character` on the class line: the child has everything the parent has and adds its own. One parent, no mixins; it compiles to `is_a`.
 - **`#` describes, `?` doubts.** A description ends up in generated documentation. A doubt never does.
+- **Names are ASCII.** `A`–`Z`, `a`–`z`, `0`–`9`, and nothing else: a name becomes an identifier and a URI in LinkML, a class in generated Python, a node in a diagram, and none of those take the rest of the alphabet. The words of the domain, in whatever alphabet they are spoken, go in the `#` description ([SPEC §4](docs/SPEC.md#4-grammar)).
 - **Every line stands on its own.** A half-typed line breaks that line and nothing else, so the diagram keeps rendering while you type.
 - **Everything you can write compiles to valid LinkML.** Skiss is a strict subset. Nothing is lost on the way up.
 
@@ -69,9 +70,11 @@ Or without installing: `npx @eriknaslund/skiss diagram model.skiss`.
 
 `skiss import schema.linkml.yaml` goes the other way: it reads a LinkML schema, YAML or JSON, and writes Skiss. What the projection could not carry — patterns, `required`, `mixins`, the names it had to convert — is one line on standard error before the diagnostics, because a sketch is a smaller language than LinkML and that line says by how much. `-o`, `-` and the diagnostics work as they do for `compile`; `--strict` exits 1 when the projected sketch has a diagnostic, and also when anything was dropped. An input that is not a schema at all — not YAML, or YAML no schema can be read out of — produces no sketch, says why on standard error, and exits 2, the code every command uses for unreadable input.
 
-`npm install @eriknaslund/skiss` also gives you the library: `compile`, `formatDiagnostic`, `parse`, `resolve`, `toLinkML`, `serialize`, `toMermaid`, `toSkiss`, `fromLinkML`, `formatDropped`, `importLinkML` and `VERSION`, browser-safe.
+`npm install @eriknaslund/skiss` also gives you the library: `compile`, `formatDiagnostic`, `parse`, `resolve`, `toLinkML`, `serialize`, `toMermaid`, `toSkiss`, `fromLinkML`, `formatDropped`, `importLinkML`, `tokenizeLine` and `VERSION`, browser-safe.
 
-Editors that render Skiss live are separate projects built on this package. This repository is the language, the library and the command line.
+`tokenizeLine` is there for editors: it says where the words of one line are — `{ kind, from, to }` for each run of characters, so a host can colour them. It is line-based and stateless, as the language is: one line in, its tokens out, nothing carried from the line before, and nothing said about a run it does not recognise, so a half-typed line keeps the colours it has earned. It is not a parser and reports nothing; diagnostics stay `parse` and `resolve`'s ([ADR 0010](docs/adr/0010-tokeniser-in-the-package.md)).
+
+Editors that render Skiss live are separate projects built on this package. This repository is the language, the library and the command line. To try the language without installing anything, open the [playground](https://erik-naslund.github.io/skiss-playground/): write a sketch, watch the diagram, share a link. Inside Obsidian, the [obsidian-skiss](https://github.com/erik-naslund/obsidian-skiss) plugin renders `skiss` code blocks in your notes.
 
 ## Where to look
 
@@ -87,9 +90,9 @@ Editors that render Skiss live are separate projects built on this package. This
 
 ## Status
 
-0.5.0, published on npm as [`@eriknaslund/skiss`](https://www.npmjs.com/package/@eriknaslund/skiss). The language parses, resolves and diagnoses; `skiss diagram` draws it, `skiss compile` writes LinkML that the real LinkML toolchain validates in CI, and `skiss import` reads a LinkML schema back into a sketch. The library exports the same in both directions and is browser-safe.
+0.6.1, published on npm as [`@eriknaslund/skiss`](https://www.npmjs.com/package/@eriknaslund/skiss). The language parses, resolves and diagnoses; `skiss diagram` draws it, `skiss compile` writes LinkML that the real LinkML toolchain validates in CI, and `skiss import` reads a LinkML schema back into a sketch. The library exports the same in both directions and is browser-safe.
 
-The [specification](docs/SPEC.md) is at 0.3 and moves when the language does, which is not with every release. Not built: editing a projected schema so that what Skiss cannot express survives the edit (SPEC §8.1). [CHANGELOG.md](CHANGELOG.md) is what changed when.
+The [specification](docs/SPEC.md) is at 0.3.1 and moves when the language does, which is not with every release. Not built: editing a projected schema so that what Skiss cannot express survives the edit (SPEC §8.1). [CHANGELOG.md](CHANGELOG.md) is what changed when.
 
 ## License
 

package/dist/cli.js

@@ -94,6 +94,21 @@ var CLASS_NAME = /^[A-Z][A-Za-z0-9]*$/;
 var FIELD_NAME = /^[a-z][A-Za-z0-9]*$/;
 var SYSTEM_NAME = /^[A-Za-z][A-Za-z0-9_-]*$/;
 var ENUM_VALUE = /^[A-Za-z0-9][A-Za-z0-9_-]*$/;
+var NAME_CHAR = /^[A-Za-z0-9]$/;
+var WORD_CHAR = /^[A-Za-z0-9_-]$/;
+var CLASS_RULE = "UpperCamelCase in ASCII letters and digits (A-Z, a-z, 0-9), uppercase first";
+var FIELD_RULE = "lowerCamelCase in ASCII letters and digits (a-z, A-Z, 0-9), lowercase first";
+function nameFault(text, allowed) {
+  for (const ch of text) {
+    if (!allowed.test(ch)) return `\`${ch}\` is not one of them.`;
+  }
+  const first = text.slice(0, 1);
+  const kind = /[A-Z]/.test(first) ? "an uppercase letter" : /[a-z]/.test(first) ? "a lowercase letter" : /[0-9]/.test(first) ? "a digit" : `\`${first}\``;
+  return `\`${text}\` starts with ${kind}.`;
+}
+function badName(rule, text, allowed = NAME_CHAR) {
+  return `${rule}: \`${text}\`. ${nameFault(text, allowed)}`;
+}
 var PRIMITIVES = {
   string: "string",
   int: "int",
@@ -109,6 +124,7 @@ var PRIMITIVES = {
 function primitiveFor(word) {
   return Object.hasOwn(PRIMITIVES, word) ? PRIMITIVES[word] : void 0;
 }
+var DROPPED = "dropped-class";
 var FIELD_ORDER = "name, `*`, `: type`, `@System`, `= Class.field`, then `#` and `?`";
 var CLASS_ORDER = "name, `< Parent`, `@System`, `~ Class`, then `#` and `?`";
 function parse(source) {
@@ -136,8 +152,8 @@ function parse(source) {
         continue;
       }
       const result = parseFieldLine(text, line);
-      if (result.ok) current.fields.push(result.value);
-      else diagnostics.push(result.diagnostic);
+      if (!result.ok) diagnostics.push(result.diagnostic);
+      else if (current !== DROPPED) current.fields.push(result.value);
       continue;
     }
     if (/\p{L}/u.test(first)) {
@@ -147,7 +163,7 @@ function parse(source) {
         current = result.value;
       } else {
         diagnostics.push(result.diagnostic);
-        current = void 0;
+        current = result.atName ? DROPPED : void 0;
       }
       continue;
     }
@@ -197,6 +213,10 @@ function splitTrailer(text) {
   return trailer;
 }
 var WORD = /[\p{L}\p{N}_-]+/uy;
+function wordAt(text, i) {
+  WORD.lastIndex = i;
+  return WORD.exec(text)?.[0];
+}
 var OPERATORS = ["*", ":", "@", "~", "=", ".", "|", "<", ",", "["];
 function tokenize(head) {
   const tokens = [];
@@ -207,11 +227,10 @@ function tokenize(head) {
       i++;
       continue;
     }
-    WORD.lastIndex = i;
-    const word = WORD.exec(head);
-    if (word !== null) {
-      tokens.push({ kind: "word", text: word[0], col: i, end: i + word[0].length });
-      i += word[0].length;
+    const word = wordAt(head, i);
+    if (word !== void 0) {
+      tokens.push({ kind: "word", text: word, col: i, end: i + word.length });
+      i += word.length;
       continue;
     }
     if (c === "[" && head[i + 1] === "]") {
@@ -233,10 +252,11 @@ function take(c) {
   return t;
 }
 var name = (t, line) => ({ text: t.text, line, col: t.col, end: t.end });
-function fail(code, message, line, span) {
+function fail(code, message, line, span, atName = false) {
   return {
     ok: false,
-    diagnostic: { severity: "error", code, message, line, col: span.col, end: span.end }
+    diagnostic: { severity: "error", code, message, line, col: span.col, end: span.end },
+    atName
   };
 }
 var afterSpan = (after) => ({ col: after.end, end: after.end });
@@ -245,17 +265,21 @@ function parseClassLine(text, line) {
   const c = { tokens: tokenize(trailer.head), pos: 0, line };
   const head = take(c);
   if (head === void 0 || head.kind !== "word") {
-    return fail("E_UNPARSABLE", "A class line must start with a class name", line, {
-      col: 0,
-      end: Math.max(1, trailer.head.length)
-    });
+    return fail(
+      "E_UNPARSABLE",
+      "A class line must start with a class name",
+      line,
+      { col: 0, end: Math.max(1, trailer.head.length) },
+      true
+    );
   }
   if (!CLASS_NAME.test(head.text)) {
     return fail(
       "E_BAD_NAME",
-      `Class names are UpperCamelCase (letters and digits, uppercase first): \`${head.text}\``,
+      badName(`Class names are ${CLASS_RULE}`, head.text),
       line,
-      head
+      head,
+      true
     );
   }
   const node = { name: name(head, line), line };
@@ -288,9 +312,10 @@ function parseClassLine(text, line) {
     if (!CLASS_NAME.test(target.text)) {
       return fail(
         "E_BAD_NAME",
-        `The class after \`~\` must be UpperCamelCase: \`${target.text}\``,
+        badName(`The class after \`~\` must be ${CLASS_RULE}`, target.text),
         line,
-        target
+        target,
+        true
       );
     }
     node.similarTo = name(target, line);
@@ -312,12 +337,7 @@ function parseFieldLine(text, line) {
   }
   if (head.kind !== "word") return unexpected(head, line, "field", FIELD_ORDER);
   if (!FIELD_NAME.test(head.text)) {
-    return fail(
-      "E_BAD_NAME",
-      `Field names are lowerCamelCase (letters and digits, lowercase first): \`${head.text}\``,
-      line,
-      head
-    );
+    return fail("E_BAD_NAME", badName(`Field names are ${FIELD_RULE}`, head.text), line, head);
   }
   const node = { name: name(head, line), line, identifier: false };
   const star = peek(c);
@@ -356,7 +376,7 @@ function parseFieldLine(text, line) {
     if (!CLASS_NAME.test(className.text)) {
       return fail(
         "E_BAD_NAME",
-        `The class after \`=\` must be UpperCamelCase: \`${className.text}\``,
+        badName(`The class after \`=\` must be ${CLASS_RULE}`, className.text),
         line,
         className
       );
@@ -382,7 +402,7 @@ function parseFieldLine(text, line) {
     if (!FIELD_NAME.test(fieldName.text)) {
       return fail(
         "E_BAD_NAME",
-        `The field after \`= ${className.text}.\` must be lowerCamelCase: \`${fieldName.text}\``,
+        badName(`The field after \`= ${className.text}.\` must be ${FIELD_RULE}`, fieldName.text),
         line,
         fieldName
       );
@@ -406,8 +426,8 @@ function parseParent(c, lt) {
     );
   }
   if (!CLASS_NAME.test(parent.text)) {
-    const message = Object.hasOwn(PRIMITIVES, parent.text) ? `\`${parent.text}\` is a primitive, not a class; a parent is a class` : `The class after \`<\` must be UpperCamelCase: \`${parent.text}\``;
-    return fail("E_BAD_NAME", message, c.line, parent);
+    const message = Object.hasOwn(PRIMITIVES, parent.text) ? `\`${parent.text}\` is a primitive, not a class; a parent is a class` : badName(`The class after \`<\` must be ${CLASS_RULE}`, parent.text);
+    return fail("E_BAD_NAME", message, c.line, parent, true);
   }
   const next = peek(c);
   if (next?.kind === "," || next?.kind === "<") {
@@ -433,7 +453,11 @@ function parseSystem(c, at) {
   if (!SYSTEM_NAME.test(system.text)) {
     return fail(
       "E_UNPARSABLE",
-      `A system name is letters, digits, \`-\` and \`_\`, starting with a letter: \`${system.text}\``,
+      badName(
+        "A system name is ASCII letters, digits, `-` and `_` (A-Z, a-z, 0-9), starting with a letter",
+        system.text,
+        WORD_CHAR
+      ),
       c.line,
       system
     );
@@ -491,7 +515,11 @@ function parseType(c, colon) {
       if (!ENUM_VALUE.test(w.text)) {
         return fail(
           "E_UNPARSABLE",
-          `An enum value is letters, digits, \`-\` and \`_\`, and does not start with \`-\` or \`_\`: \`${w.text}\``,
+          badName(
+            "An enum value is ASCII letters, digits, `-` and `_` (A-Z, a-z, 0-9), and does not start with `-` or `_`",
+            w.text,
+            WORD_CHAR
+          ),
           c.line,
           w
         );
@@ -511,7 +539,7 @@ function parseType(c, colon) {
     if (!CLASS_NAME.test(word)) {
       return fail(
         "E_BAD_NAME",
-        `A referenced class must be UpperCamelCase: \`${word}\``,
+        badName(`A referenced class must be ${CLASS_RULE}`, word),
         c.line,
         first
       );
@@ -1667,7 +1695,7 @@ function notYAML(error) {
 }
 
 // src/index.ts
-var VERSION = "0.5.0";
+var VERSION = "0.6.1";
 
 // src/cli.ts
 var HELP = `Usage: skiss <command> [options]

package/dist/index.d.ts

@@ -205,6 +205,51 @@ declare function formatDropped(dropped: Dropped[]): string;
 
 declare function toSkiss(doc: Document): string;
 
+/**
+ * Where the words of a Skiss line are, so an editor can colour them.
+ *
+ * This is not a parser and holds no language logic (AGENTS.md §2): nothing
+ * here decides whether a line is valid, what it means or what it compiles to.
+ * It reads the shapes of SPEC §3 and §4 far enough to say "this run of
+ * characters is a class name, that one is an operator", and says nothing at
+ * all about a run it does not recognise, which is what leaves a half-typed
+ * line looking calm instead of wrong. Diagnostics stay the compiler's, and
+ * reach the editor through the gutter.
+ *
+ * Every line is read on its own, as the language is (SPEC §4): no state is
+ * carried from one line to the next, so an edit is never more than one line
+ * of work.
+ *
+ * The productions come from `parse.ts` — the name shapes, the primitive
+ * table, what a word is and where a trailer starts — so that the colours and
+ * the compiler can never disagree about them (ADR 0010).
+ */
+/** What a run of characters is, as far as colour is concerned. */
+type TokenKind = 'class' | 'field' | 'system' | 'primitive'
+/** A lowercase word after `:` that is no primitive. The compiler warns; the colour does not. */
+ | 'type' | 'enum'
+/** The operators of SPEC §4, `<` included, and the `.` of a `= Class.field` join. */
+ | 'operator'
+/** The `*` that marks the identifier. */
+ | 'marker'
+/** A `#` trailer, the `#` included. */
+ | 'description'
+/** A `?` trailer, the `?` included. */
+ | 'doubt'
+/** A whole `#` line at column 0. */
+ | 'comment';
+/** A run of one line, from `from` up to but not including `to`. */
+interface Token {
+    kind: TokenKind;
+    from: number;
+    to: number;
+}
+/**
+ * The tokens of one line, in the order they appear, never overlapping. A blank
+ * line, and a line of nothing the tokenizer recognises, has none.
+ */
+declare function tokenizeLine(line: string): Token[];
+
 interface ImportResult {
     /** The projected sketch as canonical Skiss text, `\n`-terminated. */
     output: string;
@@ -223,6 +268,6 @@ interface ResolvedDocument extends Document {
 }
 declare function resolve(doc: Document): ResolvedDocument;
 
-declare const VERSION = "0.5.0";
+declare const VERSION = "0.6.1";
 
-export { type ClassNode, type CompileOptions, type CompileResult, type Diagnostic, type DiagnosticCode, type Document, type Dropped, type FieldNode, type FromLinkMLResult, type ImportResult, type JoinRef, type LinkMLAnnotations, type LinkMLAttribute, type LinkMLClass, type LinkMLCompileOptions, type LinkMLEnum, type LinkMLOptions, type LinkMLSchema, type MermaidCompileOptions, type MermaidOptions, type Name, type Primitive, type ResolvedDocument, type SerializeFormat, type Severity, type TypeRef, VERSION, compile, formatDiagnostic, formatDropped, fromLinkML, importLinkML, parse, resolve, serialize, toLinkML, toMermaid, toSkiss };
+export { type ClassNode, type CompileOptions, type CompileResult, type Diagnostic, type DiagnosticCode, type Document, type Dropped, type FieldNode, type FromLinkMLResult, type ImportResult, type JoinRef, type LinkMLAnnotations, type LinkMLAttribute, type LinkMLClass, type LinkMLCompileOptions, type LinkMLEnum, type LinkMLOptions, type LinkMLSchema, type MermaidCompileOptions, type MermaidOptions, type Name, type Primitive, type ResolvedDocument, type SerializeFormat, type Severity, type Token, type TokenKind, type TypeRef, VERSION, compile, formatDiagnostic, formatDropped, fromLinkML, importLinkML, parse, resolve, serialize, toLinkML, toMermaid, toSkiss, tokenizeLine };

package/dist/index.js

@@ -84,6 +84,21 @@ var CLASS_NAME = /^[A-Z][A-Za-z0-9]*$/;
 var FIELD_NAME = /^[a-z][A-Za-z0-9]*$/;
 var SYSTEM_NAME = /^[A-Za-z][A-Za-z0-9_-]*$/;
 var ENUM_VALUE = /^[A-Za-z0-9][A-Za-z0-9_-]*$/;
+var NAME_CHAR = /^[A-Za-z0-9]$/;
+var WORD_CHAR = /^[A-Za-z0-9_-]$/;
+var CLASS_RULE = "UpperCamelCase in ASCII letters and digits (A-Z, a-z, 0-9), uppercase first";
+var FIELD_RULE = "lowerCamelCase in ASCII letters and digits (a-z, A-Z, 0-9), lowercase first";
+function nameFault(text, allowed) {
+  for (const ch of text) {
+    if (!allowed.test(ch)) return `\`${ch}\` is not one of them.`;
+  }
+  const first = text.slice(0, 1);
+  const kind = /[A-Z]/.test(first) ? "an uppercase letter" : /[a-z]/.test(first) ? "a lowercase letter" : /[0-9]/.test(first) ? "a digit" : `\`${first}\``;
+  return `\`${text}\` starts with ${kind}.`;
+}
+function badName(rule, text, allowed = NAME_CHAR) {
+  return `${rule}: \`${text}\`. ${nameFault(text, allowed)}`;
+}
 var PRIMITIVES = {
   string: "string",
   int: "int",
@@ -99,6 +114,7 @@ var PRIMITIVES = {
 function primitiveFor(word) {
   return Object.hasOwn(PRIMITIVES, word) ? PRIMITIVES[word] : void 0;
 }
+var DROPPED = "dropped-class";
 var FIELD_ORDER = "name, `*`, `: type`, `@System`, `= Class.field`, then `#` and `?`";
 var CLASS_ORDER = "name, `< Parent`, `@System`, `~ Class`, then `#` and `?`";
 function parse(source) {
@@ -126,8 +142,8 @@ function parse(source) {
         continue;
       }
       const result = parseFieldLine(text, line);
-      if (result.ok) current.fields.push(result.value);
-      else diagnostics.push(result.diagnostic);
+      if (!result.ok) diagnostics.push(result.diagnostic);
+      else if (current !== DROPPED) current.fields.push(result.value);
       continue;
     }
     if (/\p{L}/u.test(first)) {
@@ -137,7 +153,7 @@ function parse(source) {
         current = result.value;
       } else {
         diagnostics.push(result.diagnostic);
-        current = void 0;
+        current = result.atName ? DROPPED : void 0;
       }
       continue;
     }
@@ -187,6 +203,10 @@ function splitTrailer(text) {
   return trailer;
 }
 var WORD = /[\p{L}\p{N}_-]+/uy;
+function wordAt(text, i) {
+  WORD.lastIndex = i;
+  return WORD.exec(text)?.[0];
+}
 var OPERATORS = ["*", ":", "@", "~", "=", ".", "|", "<", ",", "["];
 function tokenize(head) {
   const tokens = [];
@@ -197,11 +217,10 @@ function tokenize(head) {
       i++;
       continue;
     }
-    WORD.lastIndex = i;
-    const word = WORD.exec(head);
-    if (word !== null) {
-      tokens.push({ kind: "word", text: word[0], col: i, end: i + word[0].length });
-      i += word[0].length;
+    const word = wordAt(head, i);
+    if (word !== void 0) {
+      tokens.push({ kind: "word", text: word, col: i, end: i + word.length });
+      i += word.length;
       continue;
     }
     if (c === "[" && head[i + 1] === "]") {
@@ -223,10 +242,11 @@ function take(c) {
   return t;
 }
 var name = (t, line) => ({ text: t.text, line, col: t.col, end: t.end });
-function fail(code, message, line, span) {
+function fail(code, message, line, span, atName = false) {
   return {
     ok: false,
-    diagnostic: { severity: "error", code, message, line, col: span.col, end: span.end }
+    diagnostic: { severity: "error", code, message, line, col: span.col, end: span.end },
+    atName
   };
 }
 var afterSpan = (after) => ({ col: after.end, end: after.end });
@@ -235,17 +255,21 @@ function parseClassLine(text, line) {
   const c = { tokens: tokenize(trailer.head), pos: 0, line };
   const head = take(c);
   if (head === void 0 || head.kind !== "word") {
-    return fail("E_UNPARSABLE", "A class line must start with a class name", line, {
-      col: 0,
-      end: Math.max(1, trailer.head.length)
-    });
+    return fail(
+      "E_UNPARSABLE",
+      "A class line must start with a class name",
+      line,
+      { col: 0, end: Math.max(1, trailer.head.length) },
+      true
+    );
   }
   if (!CLASS_NAME.test(head.text)) {
     return fail(
       "E_BAD_NAME",
-      `Class names are UpperCamelCase (letters and digits, uppercase first): \`${head.text}\``,
+      badName(`Class names are ${CLASS_RULE}`, head.text),
       line,
-      head
+      head,
+      true
     );
   }
   const node = { name: name(head, line), line };
@@ -278,9 +302,10 @@ function parseClassLine(text, line) {
     if (!CLASS_NAME.test(target.text)) {
       return fail(
         "E_BAD_NAME",
-        `The class after \`~\` must be UpperCamelCase: \`${target.text}\``,
+        badName(`The class after \`~\` must be ${CLASS_RULE}`, target.text),
         line,
-        target
+        target,
+        true
       );
     }
     node.similarTo = name(target, line);
@@ -302,12 +327,7 @@ function parseFieldLine(text, line) {
   }
   if (head.kind !== "word") return unexpected(head, line, "field", FIELD_ORDER);
   if (!FIELD_NAME.test(head.text)) {
-    return fail(
-      "E_BAD_NAME",
-      `Field names are lowerCamelCase (letters and digits, lowercase first): \`${head.text}\``,
-      line,
-      head
-    );
+    return fail("E_BAD_NAME", badName(`Field names are ${FIELD_RULE}`, head.text), line, head);
   }
   const node = { name: name(head, line), line, identifier: false };
   const star = peek(c);
@@ -346,7 +366,7 @@ function parseFieldLine(text, line) {
     if (!CLASS_NAME.test(className.text)) {
       return fail(
         "E_BAD_NAME",
-        `The class after \`=\` must be UpperCamelCase: \`${className.text}\``,
+        badName(`The class after \`=\` must be ${CLASS_RULE}`, className.text),
         line,
         className
       );
@@ -372,7 +392,7 @@ function parseFieldLine(text, line) {
     if (!FIELD_NAME.test(fieldName.text)) {
       return fail(
         "E_BAD_NAME",
-        `The field after \`= ${className.text}.\` must be lowerCamelCase: \`${fieldName.text}\``,
+        badName(`The field after \`= ${className.text}.\` must be ${FIELD_RULE}`, fieldName.text),
         line,
         fieldName
       );
@@ -396,8 +416,8 @@ function parseParent(c, lt) {
     );
   }
   if (!CLASS_NAME.test(parent.text)) {
-    const message = Object.hasOwn(PRIMITIVES, parent.text) ? `\`${parent.text}\` is a primitive, not a class; a parent is a class` : `The class after \`<\` must be UpperCamelCase: \`${parent.text}\``;
-    return fail("E_BAD_NAME", message, c.line, parent);
+    const message = Object.hasOwn(PRIMITIVES, parent.text) ? `\`${parent.text}\` is a primitive, not a class; a parent is a class` : badName(`The class after \`<\` must be ${CLASS_RULE}`, parent.text);
+    return fail("E_BAD_NAME", message, c.line, parent, true);
   }
   const next = peek(c);
   if (next?.kind === "," || next?.kind === "<") {
@@ -423,7 +443,11 @@ function parseSystem(c, at) {
   if (!SYSTEM_NAME.test(system.text)) {
     return fail(
       "E_UNPARSABLE",
-      `A system name is letters, digits, \`-\` and \`_\`, starting with a letter: \`${system.text}\``,
+      badName(
+        "A system name is ASCII letters, digits, `-` and `_` (A-Z, a-z, 0-9), starting with a letter",
+        system.text,
+        WORD_CHAR
+      ),
       c.line,
       system
     );
@@ -481,7 +505,11 @@ function parseType(c, colon) {
       if (!ENUM_VALUE.test(w.text)) {
         return fail(
           "E_UNPARSABLE",
-          `An enum value is letters, digits, \`-\` and \`_\`, and does not start with \`-\` or \`_\`: \`${w.text}\``,
+          badName(
+            "An enum value is ASCII letters, digits, `-` and `_` (A-Z, a-z, 0-9), and does not start with `-` or `_`",
+            w.text,
+            WORD_CHAR
+          ),
           c.line,
           w
         );
@@ -501,7 +529,7 @@ function parseType(c, colon) {
     if (!CLASS_NAME.test(word)) {
       return fail(
         "E_BAD_NAME",
-        `A referenced class must be UpperCamelCase: \`${word}\``,
+        badName(`A referenced class must be ${CLASS_RULE}`, word),
         c.line,
         first
       );
@@ -518,36 +546,36 @@ function parseType(c, colon) {
     first
   );
 }
-function unexpected(token, line, kind, order) {
+function unexpected(token2, line, kind, order) {
   const modifiers = kind === "field" ? ["*", ":", "@", "="] : ["<", "@", "~"];
-  if (modifiers.includes(token.kind)) {
+  if (modifiers.includes(token2.kind)) {
     return fail(
       "E_UNPARSABLE",
-      `\`${token.text}\` is out of order; a ${kind} line reads: ${order}`,
+      `\`${token2.text}\` is out of order; a ${kind} line reads: ${order}`,
       line,
-      token
+      token2
     );
   }
-  if (token.kind === "<") {
+  if (token2.kind === "<") {
     return fail(
       "E_UNPARSABLE",
       "`<` marks inheritance and belongs on the class line: `Child < Parent`",
       line,
-      token
+      token2
     );
   }
-  if (token.text === "?") {
+  if (token2.text === "?") {
     return fail(
       "E_UNPARSABLE",
       "`?` attached to a word is not a marker; a doubt needs whitespace before `?` and after it",
       line,
-      token
+      token2
     );
   }
-  if (token.text === "#") {
-    return fail("E_UNPARSABLE", "A description needs whitespace before `#`", line, token);
+  if (token2.text === "#") {
+    return fail("E_UNPARSABLE", "A description needs whitespace before `#`", line, token2);
   }
-  return fail("E_UNPARSABLE", `Unexpected \`${token.text}\` on a ${kind} line`, line, token);
+  return fail("E_UNPARSABLE", `Unexpected \`${token2.text}\` on a ${kind} line`, line, token2);
 }
 
 // src/resolve.ts
@@ -1633,6 +1661,204 @@ var nothing = (element, detail) => ({
   dropped: [{ kind: UNREADABLE, element, detail }]
 });
 
+// src/highlight.ts
+var TRAILING_WS = /[ \t]+$/;
+function tokenizeLine(line) {
+  const text = line.replace(TRAILING_WS, "");
+  const first = text[0];
+  if (first === void 0) {
+    return [];
+  }
+  if (first === "#") {
+    return [{ kind: "comment", from: 0, to: text.length }];
+  }
+  const { head, trailers } = splitTrailer2(text);
+  return [...headTokens(head, isWs(first)), ...trailers];
+}
+function splitTrailer2(text) {
+  let start = -1;
+  for (let i = 1; i < text.length; i++) {
+    if (!isWs(text[i - 1])) {
+      continue;
+    }
+    if (text[i] === "#" || isDoubtMarker(text, i)) {
+      start = i;
+      break;
+    }
+  }
+  if (start < 0) {
+    return { head: text, trailers: [] };
+  }
+  const head = text.slice(0, start);
+  if (text[start] === "?") {
+    return { head, trailers: [{ kind: "doubt", from: start, to: text.length }] };
+  }
+  let doubt = -1;
+  for (let i = start + 1; i < text.length; i++) {
+    if (isDoubtMarker(text, i)) {
+      doubt = i;
+      break;
+    }
+  }
+  if (doubt < 0) {
+    return { head, trailers: [{ kind: "description", from: start, to: text.length }] };
+  }
+  return {
+    head,
+    trailers: [
+      // The whitespace between the two belongs to neither.
+      {
+        kind: "description",
+        from: start,
+        to: text.slice(0, doubt).replace(TRAILING_WS, "").length
+      },
+      { kind: "doubt", from: doubt, to: text.length }
+    ]
+  };
+}
+function scan(head) {
+  const raws = [];
+  let i = 0;
+  while (i < head.length) {
+    const c = head[i] ?? "";
+    if (isWs(c)) {
+      i += 1;
+      continue;
+    }
+    const word = wordAt(head, i);
+    if (word !== void 0) {
+      raws.push({ text: word, from: i, to: i + word.length, word: true });
+      i += word.length;
+      continue;
+    }
+    if (c === "[" && head[i + 1] === "]") {
+      raws.push({ text: "[]", from: i, to: i + 2, word: false });
+      i += 2;
+      continue;
+    }
+    const ch = String.fromCodePoint(head.codePointAt(i) ?? 0);
+    raws.push({ text: ch, from: i, to: i + ch.length, word: false });
+    i += ch.length;
+  }
+  return raws;
+}
+var token = (kind, raw) => ({ kind, from: raw.from, to: raw.to });
+function headTokens(head, field) {
+  const raws = scan(head);
+  const tokens = [];
+  let i = 0;
+  const name2 = raws[0];
+  if (name2?.word === true && (field ? FIELD_NAME : CLASS_NAME).test(name2.text)) {
+    tokens.push(token(field ? "field" : "class", name2));
+    i = 1;
+  }
+  while (i < raws.length) {
+    const raw = raws[i];
+    i += 1;
+    if (raw === void 0 || raw.word) {
+      continue;
+    }
+    switch (raw.text) {
+      case "*":
+        tokens.push(token("marker", raw));
+        break;
+      case ":":
+        tokens.push(token("operator", raw));
+        i = typeTokens(raws, i, tokens);
+        break;
+      case "@":
+        tokens.push(token("operator", raw));
+        i = nameToken(raws, i, tokens, "system", SYSTEM_NAME);
+        break;
+      case "~":
+      case "<":
+        tokens.push(token("operator", raw));
+        i = nameToken(raws, i, tokens, "class", CLASS_NAME);
+        break;
+      case "=":
+        tokens.push(token("operator", raw));
+        i = joinTokens(raws, i, tokens);
+        break;
+      case "|":
+      case "[]":
+        tokens.push(token("operator", raw));
+        break;
+      default:
+        break;
+    }
+  }
+  return tokens;
+}
+function nameToken(raws, i, tokens, kind, shape) {
+  const raw = raws[i];
+  if (raw === void 0 || !raw.word) {
+    return i;
+  }
+  if (shape.test(raw.text)) {
+    tokens.push(token(kind, raw));
+  }
+  return i + 1;
+}
+function joinTokens(raws, start, tokens) {
+  let i = nameToken(raws, start, tokens, "class", CLASS_NAME);
+  const dot = raws[i];
+  if (dot === void 0 || dot.text !== "." || dot.word) {
+    return i;
+  }
+  tokens.push(token("operator", dot));
+  i += 1;
+  return nameToken(raws, i, tokens, "field", FIELD_NAME);
+}
+function typeTokens(raws, start, tokens) {
+  const first = raws[start];
+  if (first === void 0 || !first.word) {
+    return start;
+  }
+  const words = [first];
+  const operators = [];
+  let pipes = 0;
+  let i = start + 1;
+  while (raws[i]?.text === "|") {
+    const pipe = raws[i];
+    if (pipe === void 0) {
+      break;
+    }
+    operators.push(pipe);
+    pipes += 1;
+    i += 1;
+    const value = raws[i];
+    if (value === void 0 || !value.word) {
+      break;
+    }
+    words.push(value);
+    i += 1;
+  }
+  const many = raws[i];
+  if (many !== void 0 && many.text === "[]" && !many.word) {
+    operators.push(many);
+    i += 1;
+  }
+  const typed = pipes > 0 ? words.filter((word) => ENUM_VALUE.test(word.text)).map((word) => token("enum", word)) : nameTypeTokens(first);
+  tokens.push(
+    ...[...typed, ...operators.map((raw) => token("operator", raw))].sort(
+      (a, b) => a.from - b.from
+    )
+  );
+  return i;
+}
+function nameTypeTokens(word) {
+  if (primitiveFor(word.text) !== void 0) {
+    return [token("primitive", word)];
+  }
+  if (CLASS_NAME.test(word.text)) {
+    return [token("class", word)];
+  }
+  if (/^[a-z]/.test(word.text)) {
+    return [token("type", word)];
+  }
+  return [];
+}
+
 // src/import.ts
 import { parse as parseYAML } from "yaml";
 function importLinkML(text) {
@@ -1657,7 +1883,7 @@ function notYAML(error) {
 }
 
 // src/index.ts
-var VERSION = "0.5.0";
+var VERSION = "0.6.1";
 export {
   VERSION,
   compile,
@@ -1670,6 +1896,7 @@ export {
   serialize,
   toLinkML,
   toMermaid,
-  toSkiss
+  toSkiss,
+  tokenizeLine
 };
 //# sourceMappingURL=index.js.map

package/package.json

@@ -1,6 +1,6 @@
 {
   "name": "@eriknaslund/skiss",
-  "version": "0.5.0",
+  "version": "0.6.1",
   "description": "A text notation for sketching data models, compiled to LinkML",
   "type": "module",
   "license": "MIT",