@toon-format/spec 1.3.2 → 1.4.0

package/tests/README.md

@@ -25,16 +25,18 @@ tests/
 │   │   ├── arrays-nested.json
 │   │   ├── arrays-objects.json
 │   │   ├── delimiters.json
-│   │   ├── normalization.json
 │   │   ├── whitespace.json
 │   │   └── options.json
 │   └── decode/             # Decoding tests (TOON → JSON)
 │       ├── primitives.json
+│       ├── numbers.json
 │       ├── objects.json
 │       ├── arrays-primitive.json
 │       ├── arrays-tabular.json
 │       ├── arrays-nested.json
 │       ├── delimiters.json
+│       ├── whitespace.json
+│       ├── root-form.json
 │       ├── validation-errors.json
 │       ├── indentation-errors.json
 │       └── blank-lines.json
@@ -47,7 +49,7 @@ All test fixtures follow a standard JSON structure defined in [`fixtures.schema.
 
 ```json
 {
-  "version": "1.3",
+  "version": "1.4",
   "category": "encode",
   "description": "Brief description of test category",
   "tests": [
@@ -88,13 +90,13 @@ All test fixtures follow a standard JSON structure defined in [`fixtures.schema.
 {
   "delimiter": ",",
   "indent": 2,
-  "lengthMarker": ""
+  "lengthMarker": "#"
 }
 ```
 
-- `delimiter`: `","` (comma, default), `"\t"` (tab), or `"|"` (pipe)
+- `delimiter`: `","` (comma, default), `"\t"` (tab), or `"|"` (pipe). Affects encoder output for multiline object values; decoders parse what's present
 - `indent`: Number of spaces per indentation level (default: `2`)
-- `lengthMarker`: `"#"` to prefix array lengths, or `""` for no marker (default: `""`)
+- `lengthMarker`: Optional. Set to `"#"` to prefix array lengths (e.g., `[#3]`). Omit this property to disable length markers
 
 #### Decoding Options
 
@@ -143,30 +145,32 @@ The fixture format is language-agnostic JSON, so you can load and iterate it usi
 
 | File | Description | Spec Sections |
 |------|-------------|---------------|
-| `primitives.json` | String, number, boolean, null encoding and escaping | §5 |
-| `objects.json` | Simple objects, nested objects, key encoding | §6 |
-| `arrays-primitive.json` | Inline primitive arrays, empty arrays | §7.1 |
-| `arrays-tabular.json` | Tabular format with header and rows | §7.2 |
-| `arrays-nested.json` | Arrays of arrays, mixed arrays | §7.3 |
-| `arrays-objects.json` | Objects as list items, complex nesting | §7 |
-| `delimiters.json` | Tab and pipe delimiter options | §8 |
-| `normalization.json` | BigInt, Date, undefined, NaN, Infinity handling | §5 |
-| `whitespace.json` | Formatting invariants and indentation | §4 |
+| `primitives.json` | String, number, boolean, null encoding and escaping | §7.1/§7.2, §2 |
+| `objects.json` | Simple objects, nested objects, key encoding | §8 (keys: §7.3/§7.1) |
+| `arrays-primitive.json` | Inline primitive arrays, empty arrays | §9.1 |
+| `arrays-tabular.json` | Tabular format with header and rows | §9.3 |
+| `arrays-nested.json` | Arrays of arrays, mixed arrays | §9.2/§9.4 |
+| `arrays-objects.json` | Objects as list items, complex nesting | §9, §10 |
+| `delimiters.json` | Tab and pipe delimiter options | §11 |
+| `whitespace.json` | Formatting invariants and indentation | §12 |
 | `options.json` | Length marker and delimiter option combinations | §3 |
 
 ### Decoding Tests (`fixtures/decode/`)
 
 | File | Description | Spec Sections |
 |------|-------------|---------------|
-| `primitives.json` | Parsing primitives, unescaping, ambiguity | §5 |
-| `objects.json` | Parsing objects, keys, nesting | §6 |
-| `arrays-primitive.json` | Inline array parsing | §7.1 |
-| `arrays-tabular.json` | Tabular format parsing | §7.2 |
-| `arrays-nested.json` | Nested and mixed array parsing | §7.3 |
-| `delimiters.json` | Delimiter detection and parsing | §8 |
-| `validation-errors.json` | Syntax errors, length mismatches, malformed input | §9 |
-| `indentation-errors.json` | Strict mode indentation validation | §9 |
-| `blank-lines.json` | Blank line handling in arrays | §9 |
+| `primitives.json` | Parsing primitives, unescaping, ambiguity | §4, §7.1/§7.4 |
+| `numbers.json` | Number edge cases, exponent forms, leading zeros | §4 |
+| `objects.json` | Parsing objects, keys, nesting | §8 (keys: §7.3/§7.1) |
+| `arrays-primitive.json` | Inline array parsing | §9.1 |
+| `arrays-tabular.json` | Tabular format parsing | §9.3 |
+| `arrays-nested.json` | Nested and mixed array parsing | §9.2/§9.4 |
+| `delimiters.json` | Delimiter detection and parsing | §11 |
+| `whitespace.json` | Whitespace tolerance and token trimming | §12 |
+| `root-form.json` | Root form detection (empty, single primitive) | §5 |
+| `validation-errors.json` | Syntax errors, length mismatches, malformed input | §14 |
+| `indentation-errors.json` | Strict mode indentation validation | §14.3, §12 |
+| `blank-lines.json` | Blank line handling in arrays | §14.4, §12 |
 
 ## Validating Fixtures
 

package/tests/fixtures/decode/arrays-nested.json

@@ -1,5 +1,5 @@
 {
-  "version": "1.3",
+  "version": "1.4",
   "category": "decode",
   "description": "Nested and mixed array decoding - list format, arrays of arrays, root arrays, mixed types",
   "tests": [
@@ -12,7 +12,7 @@
           { "id": 2, "name": "Second", "extra": true }
         ]
       },
-      "specSection": "7"
+      "specSection": "9.4"
     },
     {
       "name": "parses list arrays with empty items",
@@ -20,17 +20,36 @@
       "expected": {
         "items": ["first", "second", {}]
       },
-      "specSection": "7.3"
+      "specSection": "9.4"
     },
     {
-      "name": "parses objects with nested values inside list items",
+      "name": "parses list arrays with deeply nested objects",
+      "input": "items[2]:\n  - properties:\n      state:\n        type: string\n  - id: 2",
+      "expected": {
+        "items": [
+          {
+            "properties": {
+              "state": {
+                "type": "string"
+              }
+            }
+          },
+          {
+            "id": 2
+          }
+        ]
+      },
+      "specSection": "10"
+    },
+    {
+      "name": "parses list arrays containing objects with nested properties",
       "input": "items[1]:\n  - id: 1\n    nested:\n      x: 1",
       "expected": {
         "items": [
           { "id": 1, "nested": { "x": 1 } }
         ]
       },
-      "specSection": "7"
+      "specSection": "9.4"
     },
     {
       "name": "parses nested tabular arrays as first field on hyphen line",
@@ -46,7 +65,7 @@
           }
         ]
       },
-      "specSection": "7"
+      "specSection": "10"
     },
     {
       "name": "parses objects containing arrays (including empty arrays) in list format",
@@ -56,7 +75,7 @@
           { "name": "test", "data": [] }
         ]
       },
-      "specSection": "7"
+      "specSection": "9.4"
     },
     {
       "name": "parses arrays of arrays within objects",
@@ -66,7 +85,7 @@
           { "matrix": [[1, 2], [3, 4]], "name": "grid" }
         ]
       },
-      "specSection": "7"
+      "specSection": "9.2"
     },
     {
       "name": "parses nested arrays of primitives",
@@ -74,7 +93,7 @@
       "expected": {
         "pairs": [["a", "b"], ["c", "d"]]
       },
-      "specSection": "7.3"
+      "specSection": "9.2"
     },
     {
       "name": "parses quoted strings and mixed lengths in nested arrays",
@@ -82,7 +101,7 @@
       "expected": {
         "pairs": [["a", "b"], ["c,d", "e:f", "true"]]
       },
-      "specSection": "7.3"
+      "specSection": "9.2"
     },
     {
       "name": "parses empty inner arrays",
@@ -90,7 +109,7 @@
       "expected": {
         "pairs": [[], []]
       },
-      "specSection": "7.3"
+      "specSection": "9.2"
     },
     {
       "name": "parses mixed-length inner arrays",
@@ -98,37 +117,37 @@
       "expected": {
         "pairs": [[1], [2, 3]]
       },
-      "specSection": "7.3"
+      "specSection": "9.2"
     },
     {
       "name": "parses root arrays of primitives (inline)",
       "input": "[5]: x,y,\"true\",true,10",
       "expected": ["x", "y", "true", true, 10],
-      "specSection": "7"
+      "specSection": "9.1"
     },
     {
       "name": "parses root arrays of uniform objects in tabular format",
       "input": "[2]{id}:\n  1\n  2",
       "expected": [{ "id": 1 }, { "id": 2 }],
-      "specSection": "7.2"
+      "specSection": "9.3"
     },
     {
       "name": "parses root arrays of non-uniform objects in list format",
       "input": "[2]:\n  - id: 1\n  - id: 2\n    name: Ada",
       "expected": [{ "id": 1 }, { "id": 2, "name": "Ada" }],
-      "specSection": "7"
+      "specSection": "9.4"
     },
     {
       "name": "parses empty root arrays",
       "input": "[0]:",
       "expected": [],
-      "specSection": "7"
+      "specSection": "9.1"
     },
     {
       "name": "parses root arrays of arrays",
       "input": "[2]:\n  - [2]: 1,2\n  - [0]:",
       "expected": [[1, 2], []],
-      "specSection": "7.3"
+      "specSection": "9.2"
     },
     {
       "name": "parses complex mixed object with arrays and nested objects",
@@ -142,7 +161,7 @@
           "prefs": []
         }
       },
-      "specSection": "6"
+      "specSection": "8"
     },
     {
       "name": "parses arrays mixing primitives, objects and strings (list format)",
@@ -150,7 +169,7 @@
       "expected": {
         "items": [1, { "a": 1 }, "text"]
       },
-      "specSection": "7.3"
+      "specSection": "9.4"
     },
     {
       "name": "parses arrays mixing objects and arrays",
@@ -158,7 +177,7 @@
       "expected": {
         "items": [{ "a": 1 }, [1, 2]]
       },
-      "specSection": "7.3"
+      "specSection": "9.4"
     },
     {
       "name": "parses quoted key with list array format",
@@ -169,7 +188,7 @@
           { "id": 2 }
         ]
       },
-      "specSection": "7"
+      "specSection": "9.4"
     }
   ]
 }

package/tests/fixtures/decode/arrays-primitive.json

@@ -1,5 +1,5 @@
 {
-  "version": "1.3",
+  "version": "1.4",
   "category": "decode",
   "description": "Primitive array decoding - inline arrays of strings, numbers, booleans, quoted strings",
   "tests": [
@@ -9,7 +9,7 @@
       "expected": {
         "tags": ["reading", "gaming", "coding"]
       },
-      "specSection": "7.1"
+      "specSection": "9.1"
     },
     {
       "name": "parses number arrays inline",
@@ -17,7 +17,7 @@
       "expected": {
         "nums": [1, 2, 3]
       },
-      "specSection": "7.1"
+      "specSection": "9.1"
     },
     {
       "name": "parses mixed primitive arrays inline",
@@ -25,7 +25,7 @@
       "expected": {
         "data": ["x", "y", true, 10]
       },
-      "specSection": "7.1"
+      "specSection": "9.1"
     },
     {
       "name": "parses empty arrays",
@@ -33,7 +33,7 @@
       "expected": {
         "items": []
       },
-      "specSection": "7.1"
+      "specSection": "9.1"
     },
     {
       "name": "parses single-item array with empty string",
@@ -41,7 +41,7 @@
       "expected": {
         "items": [""]
       },
-      "specSection": "7.1"
+      "specSection": "9.1"
     },
     {
       "name": "parses multi-item array with empty string",
@@ -49,7 +49,7 @@
       "expected": {
         "items": ["a", "", "b"]
       },
-      "specSection": "7.1"
+      "specSection": "9.1"
     },
     {
       "name": "parses whitespace-only strings in arrays",
@@ -57,7 +57,7 @@
       "expected": {
         "items": [" ", "  "]
       },
-      "specSection": "7.1"
+      "specSection": "9.1"
     },
     {
       "name": "parses strings with delimiters in arrays",
@@ -65,7 +65,7 @@
       "expected": {
         "items": ["a", "b,c", "d:e"]
       },
-      "specSection": "7.1"
+      "specSection": "9.1"
     },
     {
       "name": "parses strings that look like primitives when quoted",
@@ -73,7 +73,7 @@
       "expected": {
         "items": ["x", "true", "42", "-3.14"]
       },
-      "specSection": "7.1"
+      "specSection": "9.1"
     },
     {
       "name": "parses strings with structural tokens in arrays",
@@ -81,7 +81,7 @@
       "expected": {
         "items": ["[5]", "- item", "{key}"]
       },
-      "specSection": "7.1"
+      "specSection": "9.1"
     },
     {
       "name": "parses quoted key with inline array",
@@ -89,7 +89,7 @@
       "expected": {
         "my-key": [1, 2, 3]
       },
-      "specSection": "7.1"
+      "specSection": "9.1"
     },
     {
       "name": "parses quoted key containing brackets with inline array",
@@ -97,7 +97,7 @@
       "expected": {
         "key[test]": [1, 2, 3]
       },
-      "specSection": "7.1"
+      "specSection": "9.1"
     },
     {
       "name": "parses quoted key with empty array",
@@ -105,7 +105,7 @@
       "expected": {
         "x-custom": []
       },
-      "specSection": "7.1"
+      "specSection": "9.1"
     }
   ]
 }

package/tests/fixtures/decode/arrays-tabular.json

@@ -1,5 +1,5 @@
 {
-  "version": "1.3",
+  "version": "1.4",
   "category": "decode",
   "description": "Tabular array decoding - parsing arrays of uniform objects with headers",
   "tests": [
@@ -12,7 +12,7 @@
           { "sku": "B2", "qty": 1, "price": 14.5 }
         ]
       },
-      "specSection": "7.2"
+      "specSection": "9.3"
     },
     {
       "name": "parses nulls and quoted values in tabular rows",
@@ -23,7 +23,18 @@
           { "id": 2, "value": "test" }
         ]
       },
-      "specSection": "7.2"
+      "specSection": "9.3"
+    },
+    {
+      "name": "parses quoted colon in tabular row as data",
+      "input": "items[2]{id,note}:\n  1,\"a:b\"\n  2,\"c:d\"",
+      "expected": {
+        "items": [
+          { "id": 1, "note": "a:b" },
+          { "id": 2, "note": "c:d" }
+        ]
+      },
+      "specSection": "9.3"
     },
     {
       "name": "parses quoted header keys in tabular arrays",
@@ -34,7 +45,7 @@
           { "order:id": 2, "full name": "Bob" }
         ]
       },
-      "specSection": "7.2"
+      "specSection": "9.3"
     },
     {
       "name": "parses quoted key with tabular array format",
@@ -45,7 +56,19 @@
           { "id": 2, "name": "Bob" }
         ]
       },
-      "specSection": "7.2"
+      "specSection": "9.3"
+    },
+    {
+      "name": "unquoted colon terminates tabular rows and starts key-value pair",
+      "input": "items[2]{id,name}:\n  1,Alice\n  2,Bob\ncount: 2",
+      "expected": {
+        "items": [
+          { "id": 1, "name": "Alice" },
+          { "id": 2, "name": "Bob" }
+        ],
+        "count": 2
+      },
+      "specSection": "9.3"
     }
   ]
 }

package/tests/fixtures/decode/blank-lines.json

@@ -1,5 +1,5 @@
 {
-  "version": "1.3",
+  "version": "1.4",
   "category": "decode",
   "description": "Blank line handling - strict mode errors on blank lines inside arrays, accepts blank lines outside arrays",
   "tests": [
@@ -11,7 +11,7 @@
       "options": {
         "strict": true
       },
-      "specSection": "9"
+      "specSection": "14.4"
     },
     {
       "name": "throws on blank line inside tabular array",
@@ -21,7 +21,7 @@
       "options": {
         "strict": true
       },
-      "specSection": "9"
+      "specSection": "14.4"
     },
     {
       "name": "throws on multiple blank lines inside array",
@@ -31,7 +31,7 @@
       "options": {
         "strict": true
       },
-      "specSection": "9"
+      "specSection": "14.4"
     },
     {
       "name": "throws on blank line with spaces inside array",
@@ -41,7 +41,7 @@
       "options": {
         "strict": true
       },
-      "specSection": "9"
+      "specSection": "14.4"
     },
     {
       "name": "throws on blank line in nested list array",
@@ -51,7 +51,7 @@
       "options": {
         "strict": true
       },
-      "specSection": "9"
+      "specSection": "14.4"
     },
     {
       "name": "accepts blank line between root-level fields",
@@ -63,7 +63,7 @@
       "options": {
         "strict": true
       },
-      "specSection": "9"
+      "specSection": "12"
     },
     {
       "name": "accepts trailing newline at end of file",
@@ -74,7 +74,7 @@
       "options": {
         "strict": true
       },
-      "specSection": "9"
+      "specSection": "12"
     },
     {
       "name": "accepts multiple trailing newlines",
@@ -85,7 +85,7 @@
       "options": {
         "strict": true
       },
-      "specSection": "9"
+      "specSection": "12"
     },
     {
       "name": "accepts blank line after array ends",
@@ -97,7 +97,7 @@
       "options": {
         "strict": true
       },
-      "specSection": "9"
+      "specSection": "12"
     },
     {
       "name": "accepts blank line between nested object fields",
@@ -111,7 +111,7 @@
       "options": {
         "strict": true
       },
-      "specSection": "9"
+      "specSection": "12"
     },
     {
       "name": "ignores blank lines inside list array when strict=false",
@@ -122,7 +122,7 @@
       "options": {
         "strict": false
       },
-      "specSection": "9"
+      "specSection": "12"
     },
     {
       "name": "ignores blank lines inside tabular array when strict=false",
@@ -136,7 +136,7 @@
       "options": {
         "strict": false
       },
-      "specSection": "9"
+      "specSection": "12"
     },
     {
       "name": "ignores multiple blank lines in arrays when strict=false",
@@ -147,7 +147,7 @@
       "options": {
         "strict": false
       },
-      "specSection": "9"
+      "specSection": "12"
     }
   ]
 }