@toon-format/spec 3.0.2 → 3.0.3

package/README.md

@@ -1,7 +1,7 @@
 # TOON Format Specification
 
 [![SPEC v3.0](https://img.shields.io/badge/spec-v3.0-lightgrey)](./SPEC.md)
-[![Tests](https://img.shields.io/badge/tests-355-green)](./tests/fixtures/)
+[![Tests](https://img.shields.io/badge/tests-358-green)](./tests/fixtures/)
 [![License: MIT](https://img.shields.io/badge/license-MIT-blue.svg)](./LICENSE)
 
 This repository contains the official specification for **Token-Oriented Object Notation (TOON)**, a compact, human-readable encoding of the JSON data model for LLM prompts. It provides a lossless serialization of the same objects, arrays, and primitives as JSON, but in a syntax that minimizes tokens and makes structure easy for models to follow.

package/SPEC.md

@@ -349,6 +349,8 @@ Note: The ABNF grammar above cannot enforce that the delimiter used in the field
 
 Note: The grammar above specifies header syntax. TOON's grammar is deliberately designed to prioritize human readability and token efficiency over strict LR(1) parseability. This requires some context-sensitive parsing (particularly for tabular row disambiguation in Section 9.3), which is a deliberate design tradeoff. Reference implementations demonstrate that deterministic parsing is achievable with modest lookahead.
 
+Between the closing bracket `]` of the bracket segment and the opening brace `{` of a fields segment (or the colon `:` if no fields segment is present), only whitespace MAY appear. If a decoder encounters non-whitespace content in these positions (e.g., additional bracket expressions like `[bar]`), the line MUST NOT be interpreted as an array header. Decoders SHOULD fall through to key-value parsing (Section 8), treating the entire pre-colon content as a literal key.
+
 Decoding requirements:
 - The bracket segment MUST parse as a non-negative integer length N.
 - If a trailing tab or pipe appears inside the brackets, it selects the active delimiter; otherwise comma is active.
@@ -729,6 +731,7 @@ When strict mode is enabled (default), decoders MUST error on the following cond
 - Missing colon in key context.
 - Invalid escape sequences or unterminated strings in quoted tokens.
 - Delimiter mismatch (detected via width/count checks and header scope).
+- Non-whitespace content between a valid bracket segment and the colon (or fields segment). Such lines MUST NOT be parsed as array headers. Decoders MUST NOT silently discard content in these positions.
 
 ### 14.3 Indentation Errors
 

package/package.json

@@ -1,7 +1,7 @@
 {
   "name": "@toon-format/spec",
   "type": "module",
-  "version": "3.0.2",
+  "version": "3.0.3",
   "packageManager": "pnpm@10.30.1",
   "description": "Official specification for Token-Oriented Object Notation (TOON)",
   "author": "Johann Schopplich <hello@johannschopplich.com>",

package/tests/fixtures/decode/objects.json

@@ -118,6 +118,33 @@
       },
       "specSection": "8"
     },
+    {
+      "name": "treats extra brackets after valid array segment as literal key",
+      "input": "foo[1][bar]: 10",
+      "expected": {
+        "foo[1][bar]": 10
+      },
+      "specSection": "6",
+      "note": "Non-whitespace [bar] between ] and : prevents array header interpretation"
+    },
+    {
+      "name": "treats non-integer bracket content as literal key",
+      "input": "foo[bar][1]: 20",
+      "expected": {
+        "foo[bar][1]": 20
+      },
+      "specSection": "6",
+      "note": "[bar] fails integer parsing; line is not an array header"
+    },
+    {
+      "name": "treats text between bracket segment and colon as literal key",
+      "input": "foo[2]extra: a,b",
+      "expected": {
+        "foo[2]extra": "a,b"
+      },
+      "specSection": "6",
+      "note": "Non-whitespace content between ] and : prevents array header interpretation"
+    },
     {
       "name": "parses quoted key with braces",
       "input": "\"{key}\": 5",