markdown-codec 8.0.1 → 10.0.0

package/README.md

@@ -105,11 +105,11 @@ The same round trip as a schema-validated [`z.codec()`](https://zod.dev) pair, m
 import { z } from "zod";
 import { markdownCodec, MarkdownBytesSchema } from "markdown-codec";
 
-const documentPackage = z.decode(markdownCodec, bytes); // throws if bytes are not well-formed UTF-8
+const documentPackage = z.decode(markdownCodec, bytes); // throws if the bytes are not text
 const bytes2 = z.encode(markdownCodec, documentPackage);
 ```
 
-`MarkdownBytesSchema` checks for well-formed UTF-8. The no-options form only; `readMarkdown`/`writeMarkdown` remain the entry points for an `AbortSignal` or diagnostic sink. Every construct-mapping gap reports through the sink as a stable code (e.g. `md/nested-emphasis-flattened`) — see `MarkdownDiagnosticCodes` and [Gotchas](#gotchas-and-quirks).
+`MarkdownBytesSchema` checks that the bytes are text at all, which is the only structural question markdown's lack of a magic number leaves. The character encoding is worked out from the bytes by [byte-codec](../byte-codec/README.md#text-decoding) rather than assumed to be UTF-8, so a file saved in the Windows system code page or as UTF-16 reads; binary content, and text in an encoding that cannot be placed, are refused rather than guessed at. The no-options form only; `readMarkdown`/`writeMarkdown` remain the entry points for an `AbortSignal` or diagnostic sink. Every construct-mapping gap reports through the sink as a stable code (e.g. `md/nested-emphasis-flattened`) — see `MarkdownDiagnosticCodes` and [Gotchas](#gotchas-and-quirks).
 
 ## Two encodings: `DocumentTree` and `ContentDocument`
 
@@ -229,7 +229,7 @@ Emission is the inverse and validates first: a section's markers must pair as ba
 
 ## HTML-table fallback
 
-GFM's own pipe-table syntax (github.github.com/gfm, "Tables (extension)") holds inline content only — no cell can carry `colSpan`/`rowSpan`/a background, and no cell can hold a block that isn't inlinable (a nested table, most notably). `writeMarkdownContent` detects this per table (`tableNeedsHtmlFallback`, `src/emit/html-table.ts`): the moment ANY cell needs one of these, the WHOLE table renders as a raw HTML `<table>` block instead of the plain `| a | b |` writer — GFM's own cell grammar cannot mix an HTML sub-block into one cell of an otherwise pipe-syntax table, so a partial, per-cell fallback is not an option. A raw HTML `<table>` block is legal CommonMark regardless (spec 0.31.2, "HTML blocks" start condition 6 names `table` directly — <https://spec.commonmark.org/0.31.2/#html-blocks>). A table with none of these needs still emits the existing, human-readable pipe syntax, unaffected.
+GFM's own pipe-table syntax (github.github.com/gfm, "Tables (extension)") holds inline content only — no cell can carry `colSpan`/`rowSpan`/a background, and no cell can hold a block that isn't inlinable (a nested table, most notably). `writeMarkdownContent` detects this per table (`tableNeedsHtmlFallback`, `src/emit/html-table.ts`): the moment ANY cell needs one of these, the WHOLE table renders as a raw HTML `<table>` block instead of the plain `| a | b |` writer — GFM's own cell grammar cannot mix an HTML sub-block into one cell of an otherwise pipe-syntax table, so a partial, per-cell fallback is not an option. A raw HTML `<table>` block is legal CommonMark regardless (spec 0.31.2, "HTML blocks" start condition 6 names `table` directly — <https://spec.commonmark.org/0.31.2/#html-blocks>). A table with none of these needs still emits the existing, human-readable pipe syntax, unaffected. Either way a table must obey the grid rule (document-schema.js's `findTableGridFault`: no blocks or span on a covered position, rows of one length, no region running past the grid or into another) or `writeMarkdownContent` throws `MarkdownTableGridFaultError`, since neither a pipe row nor an HTML row has a cell for a covered position's content.
 
 `readMarkdownContent`/`readMarkdown` recognise this same shape on the way back in (`parseHtmlTable`, `src/html/html-table.ts`) — the one exception to this package's own "raw HTML is never parsed as markup" rule (see the raw-HTML gotcha above and `src/html/html.ts`'s own top comment): a block-level HTML that is, in full, one well-formed `<table>` lowers straight to a real `ContentTable`, so a table written through this fallback reads back as an equal (or equivalent) `ContentTable` rather than degrading to opaque preserved text. This recognition is gated on GFM's table extension being enabled (`gfmTables`, default `true`) exactly the way plain pipe-table promotion already is — with it disabled, a `<table>` block stays exactly the opaque raw HTML it always was, since a `ContentTable` is a GFM-table-extension construct either way it is spelled in the source.
 

package/dist/block/block.d.cts

@@ -1,6 +1,6 @@
 import { s as MarkdownDocumentNode } from "../ast-CNfTgS0N.cjs";
 import { r as FootnoteLabelSet } from "../footnote-CKk4JbLk.cjs";
-import { i as MarkdownDiagnosticSink } from "../diagnostics-CjfBoW4K.cjs";
+import { i as MarkdownDiagnosticSink } from "../diagnostics-BJtTrxpp.cjs";
 import { LinkReferenceMap } from "../inline/link.cjs";
 import { t as InlineParseOptions } from "../inline-CnO86zMw.cjs";
 //#region src/block/block.d.ts

package/dist/block/block.d.ts

@@ -1,6 +1,6 @@
 import { s as MarkdownDocumentNode } from "../ast-CNfTgS0N.js";
 import { r as FootnoteLabelSet } from "../footnote-CKk4JbLk.js";
-import { i as MarkdownDiagnosticSink } from "../diagnostics-CjfBoW4K.js";
+import { i as MarkdownDiagnosticSink } from "../diagnostics-BJtTrxpp.js";
 import { LinkReferenceMap } from "../inline/link.js";
 import { t as InlineParseOptions } from "../inline-DItZ-VVw.js";
 //#region src/block/block.d.ts

package/dist/block/definitions.d.cts

@@ -1,4 +1,4 @@
-import { i as MarkdownDiagnosticSink } from "../diagnostics-CjfBoW4K.cjs";
+import { i as MarkdownDiagnosticSink } from "../diagnostics-BJtTrxpp.cjs";
 import { LinkReferenceDefinition } from "../inline/link.cjs";
 //#region src/block/definitions.d.ts
 declare function extractDefinitions(content: string, references: Map<string, LinkReferenceDefinition>, sink?: MarkdownDiagnosticSink, startLine?: number): string;

package/dist/block/definitions.d.ts

@@ -1,4 +1,4 @@
-import { i as MarkdownDiagnosticSink } from "../diagnostics-CjfBoW4K.js";
+import { i as MarkdownDiagnosticSink } from "../diagnostics-BJtTrxpp.js";
 import { LinkReferenceDefinition } from "../inline/link.js";
 //#region src/block/definitions.d.ts
 declare function extractDefinitions(content: string, references: Map<string, LinkReferenceDefinition>, sink?: MarkdownDiagnosticSink, startLine?: number): string;

package/dist/codec.cjs

@@ -3,22 +3,15 @@ const require_read = require("./read.cjs");
 const require_write = require("./write.cjs");
 let zod = require("zod");
 let document_schema_js = require("document-schema.js");
+let byte_codec = require("byte-codec");
 //#region src/codec.ts
-function isWellFormedUtf8Text(bytes) {
-	try {
-		new TextDecoder("utf-8", { fatal: true }).decode(bytes);
-		return true;
-	} catch {
-		return false;
-	}
-}
-const MarkdownBytesSchema = zod.z.instanceof(Uint8Array).refine(isWellFormedUtf8Text, { message: "not well-formed UTF-8 text" });
+const MarkdownBytesSchema = zod.z.instanceof(Uint8Array).refine((bytes) => (0, byte_codec.tryDecodeText)(bytes) !== void 0, { message: "not text in any supported character encoding" });
 const markdownCodec = zod.z.codec(MarkdownBytesSchema, document_schema_js.DocumentTreeSchema, {
-	decode: (bytes) => require_read.readMarkdown(new TextDecoder().decode(bytes)).documentPackage,
+	decode: (bytes) => require_read.readMarkdown((0, byte_codec.decodeText)(bytes).text).documentPackage,
 	encode: (documentPackage) => new TextEncoder().encode(require_write.writeMarkdown(documentPackage))
 });
 const markdownContentCodec = zod.z.codec(MarkdownBytesSchema, document_schema_js.ContentDocumentSchema, {
-	decode: (bytes) => require_read.readMarkdownContent(new TextDecoder().decode(bytes)).document,
+	decode: (bytes) => require_read.readMarkdownContent((0, byte_codec.decodeText)(bytes).text).document,
 	encode: (document) => new TextEncoder().encode(require_write.writeMarkdownContent(document))
 });
 //#endregion

package/dist/codec.d.cts

@@ -17,8 +17,8 @@ declare const markdownCodec: z.ZodCodec<z.ZodCustom<Uint8Array<ArrayBuffer>, Uin
     paragraph: z.ZodOptional<z.ZodObject<{
       alignment: z.ZodOptional<z.ZodEnum<{
         left: "left";
-        center: "center";
         right: "right";
+        center: "center";
         justify: "justify";
       }>>;
       list: z.ZodOptional<z.ZodObject<{
@@ -172,8 +172,8 @@ declare const markdownCodec: z.ZodCodec<z.ZodCustom<Uint8Array<ArrayBuffer>, Uin
     paragraph: z.ZodOptional<z.ZodObject<{
       alignment: z.ZodOptional<z.ZodEnum<{
         left: "left";
-        center: "center";
         right: "right";
+        center: "center";
         justify: "justify";
       }>>;
       list: z.ZodOptional<z.ZodObject<{
@@ -332,8 +332,8 @@ declare const markdownCodec: z.ZodCodec<z.ZodCustom<Uint8Array<ArrayBuffer>, Uin
     paragraph: z.ZodOptional<z.ZodObject<{
       alignment: z.ZodOptional<z.ZodEnum<{
         left: "left";
-        center: "center";
         right: "right";
+        center: "center";
         justify: "justify";
       }>>;
       list: z.ZodOptional<z.ZodObject<{
@@ -487,8 +487,8 @@ declare const markdownCodec: z.ZodCodec<z.ZodCustom<Uint8Array<ArrayBuffer>, Uin
     paragraph: z.ZodOptional<z.ZodObject<{
       alignment: z.ZodOptional<z.ZodEnum<{
         left: "left";
-        center: "center";
         right: "right";
+        center: "center";
         justify: "justify";
       }>>;
       list: z.ZodOptional<z.ZodObject<{
@@ -630,11 +630,11 @@ declare const markdownCodec: z.ZodCodec<z.ZodCustom<Uint8Array<ArrayBuffer>, Uin
   children: z.ZodArray<z.ZodObject<{
     origin: z.ZodOptional<z.ZodEnum<{
       chart: "chart";
-      body: "body";
       diagram: "diagram";
       table: "table";
       image: "image";
       notes: "notes";
+      body: "body";
     }>>;
     interpretation: z.ZodOptional<z.ZodObject<{
       transcript: z.ZodOptional<z.ZodObject<{
@@ -701,8 +701,8 @@ declare const markdownCodec: z.ZodCodec<z.ZodCustom<Uint8Array<ArrayBuffer>, Uin
     paragraph: z.ZodOptional<z.ZodObject<{
       alignment: z.ZodOptional<z.ZodEnum<{
         left: "left";
-        center: "center";
         right: "right";
+        center: "center";
         justify: "justify";
       }>>;
       list: z.ZodOptional<z.ZodObject<{
@@ -845,11 +845,11 @@ declare const markdownContentCodec: z.ZodCodec<z.ZodCustom<Uint8Array<ArrayBuffe
   sections: z.ZodArray<z.ZodObject<{
     origin: z.ZodOptional<z.ZodEnum<{
       chart: "chart";
-      body: "body";
       diagram: "diagram";
       table: "table";
       image: "image";
       notes: "notes";
+      body: "body";
     }>>;
     interpretation: z.ZodOptional<z.ZodObject<{
       transcript: z.ZodOptional<z.ZodObject<{
@@ -994,11 +994,11 @@ declare const markdownContentCodec: z.ZodCodec<z.ZodCustom<Uint8Array<ArrayBuffe
   slides: z.ZodArray<z.ZodObject<{
     origin: z.ZodOptional<z.ZodEnum<{
       chart: "chart";
-      body: "body";
       diagram: "diagram";
       table: "table";
       image: "image";
       notes: "notes";
+      body: "body";
     }>>;
     interpretation: z.ZodOptional<z.ZodObject<{
       transcript: z.ZodOptional<z.ZodObject<{
@@ -1027,11 +1027,11 @@ declare const markdownContentCodec: z.ZodCodec<z.ZodCustom<Uint8Array<ArrayBuffe
       blocks: z.ZodLazy<z.ZodArray<z.ZodType<import("document-schema.js").ContentBlock, import("document-schema.js").ContentBlock, z.core.$ZodTypeInternals<import("document-schema.js").ContentBlock, import("document-schema.js").ContentBlock>>>>;
       origin: z.ZodOptional<z.ZodEnum<{
         chart: "chart";
-        body: "body";
         diagram: "diagram";
         table: "table";
         image: "image";
         notes: "notes";
+        body: "body";
       }>>;
       interpretation: z.ZodOptional<z.ZodObject<{
         transcript: z.ZodOptional<z.ZodObject<{
@@ -1190,11 +1190,11 @@ declare const markdownContentCodec: z.ZodCodec<z.ZodCustom<Uint8Array<ArrayBuffe
   sheets: z.ZodArray<z.ZodObject<{
     origin: z.ZodOptional<z.ZodEnum<{
       chart: "chart";
-      body: "body";
       diagram: "diagram";
       table: "table";
       image: "image";
       notes: "notes";
+      body: "body";
     }>>;
     interpretation: z.ZodOptional<z.ZodObject<{
       transcript: z.ZodOptional<z.ZodObject<{
@@ -1219,11 +1219,11 @@ declare const markdownContentCodec: z.ZodCodec<z.ZodCustom<Uint8Array<ArrayBuffe
     cells: z.ZodArray<z.ZodObject<{
       origin: z.ZodOptional<z.ZodEnum<{
         chart: "chart";
-        body: "body";
         diagram: "diagram";
         table: "table";
         image: "image";
         notes: "notes";
+        body: "body";
       }>>;
       interpretation: z.ZodOptional<z.ZodObject<{
         transcript: z.ZodOptional<z.ZodObject<{
@@ -1299,11 +1299,11 @@ declare const markdownContentCodec: z.ZodCodec<z.ZodCustom<Uint8Array<ArrayBuffe
       runs: z.ZodOptional<z.ZodArray<z.ZodObject<{
         origin: z.ZodOptional<z.ZodEnum<{
           chart: "chart";
-          body: "body";
           diagram: "diagram";
           table: "table";
           image: "image";
           notes: "notes";
+          body: "body";
         }>>;
         interpretation: z.ZodOptional<z.ZodObject<{
           transcript: z.ZodOptional<z.ZodObject<{
@@ -1538,8 +1538,8 @@ declare const markdownContentCodec: z.ZodCodec<z.ZodCustom<Uint8Array<ArrayBuffe
       }, z.core.$strip>>;
       alignment: z.ZodOptional<z.ZodEnum<{
         left: "left";
-        center: "center";
         right: "right";
+        center: "center";
         justify: "justify";
       }>>;
       verticalAlignment: z.ZodOptional<z.ZodEnum<{
@@ -1598,11 +1598,11 @@ declare const markdownContentCodec: z.ZodCodec<z.ZodCustom<Uint8Array<ArrayBuffe
     images: z.ZodArray<z.ZodObject<{
       origin: z.ZodOptional<z.ZodEnum<{
         chart: "chart";
-        body: "body";
         diagram: "diagram";
         table: "table";
         image: "image";
         notes: "notes";
+        body: "body";
       }>>;
       interpretation: z.ZodOptional<z.ZodObject<{
         transcript: z.ZodOptional<z.ZodObject<{
@@ -1681,10 +1681,10 @@ declare const markdownContentCodec: z.ZodCodec<z.ZodCustom<Uint8Array<ArrayBuffe
           }>;
           align: z.ZodEnum<{
             left: "left";
-            center: "center";
             right: "right";
             top: "top";
             bottom: "bottom";
+            center: "center";
             inside: "inside";
             outside: "outside";
           }>;
@@ -1724,10 +1724,10 @@ declare const markdownContentCodec: z.ZodCodec<z.ZodCustom<Uint8Array<ArrayBuffe
           }>;
           align: z.ZodEnum<{
             left: "left";
-            center: "center";
             right: "right";
             top: "top";
             bottom: "bottom";
+            center: "center";
             inside: "inside";
             outside: "outside";
           }>;
@@ -1816,10 +1816,10 @@ declare const markdownContentCodec: z.ZodCodec<z.ZodCustom<Uint8Array<ArrayBuffe
         endColumn: z.ZodNumber;
       }, z.core.$strip>>;
       type: z.ZodEnum<{
-        decimal: "decimal";
-        list: "list";
         date: "date";
         custom: "custom";
+        decimal: "decimal";
+        list: "list";
         time: "time";
         whole: "whole";
         textLength: "textLength";
@@ -2292,10 +2292,10 @@ declare const markdownContentCodec: z.ZodCodec<z.ZodCustom<Uint8Array<ArrayBuffe
         value: z.ZodObject<{
           type: z.ZodEnum<{
             formula: "formula";
+            percent: "percent";
             num: "num";
-            min: "min";
             max: "max";
-            percent: "percent";
+            min: "min";
             percentile: "percentile";
           }>;
           value: z.ZodOptional<z.ZodString>;
@@ -2339,10 +2339,10 @@ declare const markdownContentCodec: z.ZodCodec<z.ZodCustom<Uint8Array<ArrayBuffe
       min: z.ZodObject<{
         type: z.ZodEnum<{
           formula: "formula";
+          percent: "percent";
           num: "num";
-          min: "min";
           max: "max";
-          percent: "percent";
+          min: "min";
           percentile: "percentile";
         }>;
         value: z.ZodOptional<z.ZodString>;
@@ -2350,10 +2350,10 @@ declare const markdownContentCodec: z.ZodCodec<z.ZodCustom<Uint8Array<ArrayBuffe
       max: z.ZodObject<{
         type: z.ZodEnum<{
           formula: "formula";
+          percent: "percent";
           num: "num";
-          min: "min";
           max: "max";
-          percent: "percent";
+          min: "min";
           percentile: "percentile";
         }>;
         value: z.ZodOptional<z.ZodString>;
@@ -2398,10 +2398,10 @@ declare const markdownContentCodec: z.ZodCodec<z.ZodCustom<Uint8Array<ArrayBuffe
       thresholds: z.ZodArray<z.ZodObject<{
         type: z.ZodEnum<{
           formula: "formula";
+          percent: "percent";
           num: "num";
-          min: "min";
           max: "max";
-          percent: "percent";
+          min: "min";
           percentile: "percentile";
         }>;
         value: z.ZodOptional<z.ZodString>;
@@ -2535,11 +2535,11 @@ declare const markdownContentCodec: z.ZodCodec<z.ZodCustom<Uint8Array<ArrayBuffe
   pages: z.ZodArray<z.ZodObject<{
     origin: z.ZodOptional<z.ZodEnum<{
       chart: "chart";
-      body: "body";
       diagram: "diagram";
       table: "table";
       image: "image";
       notes: "notes";
+      body: "body";
     }>>;
     interpretation: z.ZodOptional<z.ZodObject<{
       transcript: z.ZodOptional<z.ZodObject<{
@@ -2568,11 +2568,11 @@ declare const markdownContentCodec: z.ZodCodec<z.ZodCustom<Uint8Array<ArrayBuffe
       blocks: z.ZodLazy<z.ZodArray<z.ZodType<import("document-schema.js").ContentBlock, import("document-schema.js").ContentBlock, z.core.$ZodTypeInternals<import("document-schema.js").ContentBlock, import("document-schema.js").ContentBlock>>>>;
       origin: z.ZodOptional<z.ZodEnum<{
         chart: "chart";
-        body: "body";
         diagram: "diagram";
         table: "table";
         image: "image";
         notes: "notes";
+        body: "body";
       }>>;
       interpretation: z.ZodOptional<z.ZodObject<{
         transcript: z.ZodOptional<z.ZodObject<{
@@ -2641,11 +2641,11 @@ declare const markdownContentCodec: z.ZodCodec<z.ZodCustom<Uint8Array<ArrayBuffe
     vectors: z.ZodArray<z.ZodDiscriminatedUnion<[z.ZodObject<{
       origin: z.ZodOptional<z.ZodEnum<{
         chart: "chart";
-        body: "body";
         diagram: "diagram";
         table: "table";
         image: "image";
         notes: "notes";
+        body: "body";
       }>>;
       interpretation: z.ZodOptional<z.ZodObject<{
         transcript: z.ZodOptional<z.ZodObject<{
@@ -2779,11 +2779,11 @@ declare const markdownContentCodec: z.ZodCodec<z.ZodCustom<Uint8Array<ArrayBuffe
     }, z.core.$strip>, z.ZodObject<{
       origin: z.ZodOptional<z.ZodEnum<{
         chart: "chart";
-        body: "body";
         diagram: "diagram";
         table: "table";
         image: "image";
         notes: "notes";
+        body: "body";
       }>>;
       interpretation: z.ZodOptional<z.ZodObject<{
         transcript: z.ZodOptional<z.ZodObject<{
@@ -2917,11 +2917,11 @@ declare const markdownContentCodec: z.ZodCodec<z.ZodCustom<Uint8Array<ArrayBuffe
     }, z.core.$strip>, z.ZodObject<{
       origin: z.ZodOptional<z.ZodEnum<{
         chart: "chart";
-        body: "body";
         diagram: "diagram";
         table: "table";
         image: "image";
         notes: "notes";
+        body: "body";
       }>>;
       interpretation: z.ZodOptional<z.ZodObject<{
         transcript: z.ZodOptional<z.ZodObject<{
@@ -3005,11 +3005,11 @@ declare const markdownContentCodec: z.ZodCodec<z.ZodCustom<Uint8Array<ArrayBuffe
     }, z.core.$strip>, z.ZodObject<{
       origin: z.ZodOptional<z.ZodEnum<{
         chart: "chart";
-        body: "body";
         diagram: "diagram";
         table: "table";
         image: "image";
         notes: "notes";
+        body: "body";
       }>>;
       interpretation: z.ZodOptional<z.ZodObject<{
         transcript: z.ZodOptional<z.ZodObject<{
@@ -3265,11 +3265,11 @@ declare const markdownContentCodec: z.ZodCodec<z.ZodCustom<Uint8Array<ArrayBuffe
   formula: z.ZodObject<{
     origin: z.ZodOptional<z.ZodEnum<{
       chart: "chart";
-      body: "body";
       diagram: "diagram";
       table: "table";
       image: "image";
       notes: "notes";
+      body: "body";
     }>>;
     interpretation: z.ZodOptional<z.ZodObject<{
       transcript: z.ZodOptional<z.ZodObject<{