@toon-format/spec 1.4.0 → 1.5.0

This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
package/CHANGELOG.md CHANGED
@@ -5,6 +5,20 @@ All notable changes to the TOON specification will be documented in this file.
5
5
  The format is based on [Keep a Changelog](https://keepachangelog.com/en/1.0.0/),
6
6
  and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0.html).
7
7
 
8
+ ## [1.5] - 2025-11-08
9
+
10
+ ### Added
11
+
12
+ - Optional key folding for encoders: `keyFolding="safe"` mode with `flattenDepth` control to collapse single-key object chains into dotted-path notation (§13.4)
13
+ - Optional path expansion for decoders: `expandPaths="safe"` mode to split dotted keys into nested objects, with conflict resolution tied to `strict` option (§13.4, §14.5)
14
+ - IdentifierSegment terminology and path separator definition (fixed to `"."` in v1.5) (§1.9)
15
+ - Deep-merge semantics for path expansion: recursive merge for objects, error on conflict when `strict=true`, last-write-wins (LWW) when `strict=false` (§13.4)
16
+
17
+ ### Changed
18
+
19
+ - Both new features default to OFF and are fully backward-compatible
20
+ - Safe-mode folding requires IdentifierSegment validation, collision avoidance, and no quoting
21
+
8
22
  ## [1.4] - 2025-11-05
9
23
 
10
24
  ### Changed
package/README.md CHANGED
@@ -1,6 +1,6 @@
1
1
  # TOON Format Specification
2
2
 
3
- [![SPEC v1.4](https://img.shields.io/badge/spec-v1.4-lightgrey)](./SPEC.md)
3
+ [![SPEC v1.5](https://img.shields.io/badge/spec-v1.5-lightgrey)](./SPEC.md)
4
4
  [![Tests](https://img.shields.io/badge/tests-323-green)](./tests/fixtures/)
5
5
  [![License: MIT](https://img.shields.io/badge/license-MIT-blue.svg)](./LICENSE)
6
6
 
@@ -10,12 +10,24 @@ This repository contains the official specification for **Token-Oriented Object
10
10
 
11
11
  [→ Read the full specification (SPEC.md)](./SPEC.md)
12
12
 
13
- - **Version:** 1.4 (2025-11-05)
13
+ - **Version:** 1.5 (2025-11-10)
14
14
  - **Status:** Working Draft
15
15
  - **License:** MIT
16
16
 
17
17
  The specification includes complete grammar (ABNF), encoding rules, validation requirements, and conformance criteria.
18
18
 
19
+ ### New in v1.5
20
+
21
+ - **Key Folding** (encode): Collapse nested single-key objects into compact dotted paths
22
+ - `{"a": {"b": {"c": 1}}}` → `a.b.c: 1`
23
+ - Opt-in via `keyFolding="safe"` with `flattenDepth` control
24
+ - **Path Expansion** (decode): Expand dotted keys back to nested objects
25
+ - `a.b.c: 1` → `{"a": {"b": {"c": 1}}}`
26
+ - Opt-in via `expandPaths="safe"` with deep-merge semantics
27
+
28
+ > [!NOTE]
29
+ > Both features are opt-in to maintain backward compatibility.
30
+
19
31
  ## What is TOON?
20
32
 
21
33
  **Token-Oriented Object Notation** is a compact, human-readable serialization format designed for passing structured data to Large Language Models with significantly reduced token usage. It's intended for LLM input, not output.
package/SPEC.md CHANGED
@@ -2,9 +2,9 @@
2
2
 
3
3
  ## Token-Oriented Object Notation
4
4
 
5
- **Version:** 1.4
5
+ **Version:** 1.5
6
6
 
7
- **Date:** 2025-11-05
7
+ **Date:** 2025-11-10
8
8
 
9
9
  **Status:** Working Draft
10
10
 
@@ -189,6 +189,12 @@ Implementations that fail to conform to any MUST or REQUIRED level requirement a
189
189
  - Regular expressions appear in slash-delimited form.
190
190
  - ABNF snippets follow RFC 5234; HTAB means the U+0009 character.
191
191
 
192
+ ### 1.9 Key Folding and Path Expansion Terms
193
+
194
+ - IdentifierSegment: A key segment eligible for safe folding and expansion, matching the pattern `^[A-Za-z_][A-Za-z0-9_]*$` (contains only letters, digits, and underscores; does not start with a digit; does not contain dots).
195
+ - Path separator: The character used to join/split key segments during folding and expansion. Fixed to `"."` (U+002E, FULL STOP) in v1.5.
196
+ - Note: Unquoted keys in TOON remain permissive per §7.3 (`^[A-Za-z_][A-Za-z0-9_.]*$`, allowing dots). IdentifierSegment is a stricter pattern used only for safe folding and expansion eligibility checks.
197
+
192
198
  ## 2. Data Model
193
199
 
194
200
  - TOON models data as:
@@ -351,6 +357,8 @@ Decoding requirements:
351
357
  - If a fields segment occurs between the bracket and the colon, parse field names using the active delimiter; quoted names MUST be unescaped per Section 7.1.
352
358
  - A colon MUST follow the bracket and optional fields; missing colon MUST error.
353
359
 
360
+ Note: Key folding (§13.4) affects only the key prefix in headers. The header grammar remains unchanged. Example: `data.meta.items[2]{id,name}:` is a valid header with a folded key prefix `data.meta.items`, followed by a standard bracket segment, field list, and colon. Parsing treats folded keys as literal keys; see §13.4 for optional path expansion.
361
+
354
362
  ## 7. Strings and Keys
355
363
 
356
364
  ### 7.1 Escaping (Encoding and Decoding)
@@ -393,6 +401,8 @@ Object keys and tabular field names:
393
401
 
394
402
  Keys requiring quoting per the above rules MUST be quoted in all contexts, including array headers (e.g., "my-key"[N]:).
395
403
 
404
+ Encoders MAY perform key folding when enabled (see §13.4 for complete folding rules and requirements).
405
+
396
406
  ### 7.4 Decoding Rules for Strings and Keys (Decoding)
397
407
 
398
408
  - Quoted strings and keys MUST be unescaped per Section 7.1; any other escape MUST error. Quoted primitives remain strings.
@@ -409,6 +419,7 @@ Keys requiring quoting per the above rules MUST be quoted in all contexts, inclu
409
419
  - Nested or empty objects: key: on its own line. If non-empty, nested fields appear at depth +1.
410
420
  - Key order: Implementations MUST preserve encounter order when emitting fields.
411
421
  - An empty object at the root yields an empty document (no lines).
422
+ - Dotted keys (e.g., `user.name`) are valid literal keys in TOON. Decoders MUST treat them as single literal keys unless path expansion is explicitly enabled (see §13.4). This preserves backward compatibility and allows safe opt-in expansion behavior.
412
423
  - Decoding:
413
424
  - A line "key:" with nothing after the colon at depth d opens an object; subsequent lines at depth > d belong to that object until the depth decreases to ≤ d.
414
425
  - Lines "key: value" at the same depth are sibling fields.
@@ -582,12 +593,89 @@ Options:
582
593
  - indent (default: 2 spaces)
583
594
  - delimiter (document delimiter; default: comma; alternatives: tab, pipe)
584
595
  - lengthMarker (default: disabled)
596
+ - keyFolding (default: `"off"`; alternatives: `"safe"`)
597
+ - flattenDepth (default: Infinity when keyFolding is `"safe"`; non-negative integer ≥ 0; values 0 or 1 have no practical folding effect)
585
598
  - Decoder options:
586
599
  - indent (default: 2 spaces)
587
- - strict (default: true)
600
+ - strict (default: `true`)
601
+ - expandPaths (default: `"off"`; alternatives: `"safe"`)
588
602
 
589
603
  Strict-mode errors are enumerated in §14; validators MAY add informative diagnostics for style and encoding invariants.
590
604
 
605
+ ### 13.4 Key Folding and Path Expansion
606
+
607
+ Key folding and path expansion are optional transformations for compact dotted-path notation. Both default to `"off"`.
608
+
609
+ #### Encoder: Key Folding
610
+
611
+ Key folding allows encoders to collapse chains of single-key objects into dotted-path notation, reducing verbosity for deeply nested structures.
612
+
613
+ Mode: `"off"` | `"safe"` (default: `"off"`)
614
+ - `"off"`: No folding is performed. All objects are encoded with standard nesting.
615
+ - `"safe"`: Fold eligible chains according to the rules below.
616
+
617
+ flattenDepth: The maximum number of segments from K0 to include in the folded path (default: Infinity when keyFolding is `"safe"`; values less than 2 have no practical effect).
618
+ - A value of 2 folds only two-segment chains: `{a: {b: val}}` → `a.b: val`.
619
+ - A value of Infinity folds entire eligible chains: `{a: {b: {c: val}}}` → `a.b.c: val`.
620
+
621
+ Foldable chain: A chain K0 → K1 → ... → Kn is foldable when:
622
+ - Each Ki (where i = 0 to n−1) is an object with exactly one key Ki+1.
623
+ - The chain stops at the first non-single-key object or when encountering a leaf value.
624
+ - Arrays are not considered single-key objects; a chain stops at arrays.
625
+ - The leaf value at Kn is either a primitive, an array, or an empty object.
626
+
627
+ Safe mode requirements (all MUST hold for a chain to be folded):
628
+ 1. All folded segments K0 through K(d−1) (where d = min(chain length, flattenDepth)) MUST be IdentifierSegments (§1.9): matching `^[A-Za-z_][A-Za-z0-9_]*$`.
629
+ 2. No segment may contain the path separator (`.` in v1.5).
630
+ 3. The resulting folded key string MUST NOT equal any existing sibling literal key at the same object depth (collision avoidance).
631
+ 4. If any segment would require quoting per §7.3, the chain MUST NOT be folded.
632
+
633
+ Folding process:
634
+ - For a foldable chain of length n, determine d = min(n, flattenDepth).
635
+ - Fold segments K0 through K(d−1) into a single key: `K0.K1.....K(d−1)`.
636
+ - If d < n, emit the remaining structure (Kd through Kn) as normal nested objects.
637
+ - The leaf value at Kn is encoded normally (primitive, array, or empty object).
638
+
639
+ Examples:
640
+ - `{a: {b: {c: 1}}}` with safe mode, depth=Infinity → `a.b.c: 1`
641
+ - `{a: {b: {c: {d: 1}}}}` with safe mode, depth=2 → produces `a.b:` followed by nested `c:` and `d: 1` at appropriate depths
642
+ - `{data: {"full-name": {x: 1}}}` → safe mode skips (segment `"full-name"` requires quoting); emits standard nested structure
643
+
644
+ #### Decoder: Path Expansion
645
+
646
+ Path expansion allows decoders to split dotted keys into nested object structures, enabling round-trip compatibility with folded encodings.
647
+
648
+ Mode: `"off"` | `"safe"` (default: `"off"`)
649
+ - `"off"`: Dotted keys are treated as literal keys. No expansion is performed.
650
+ - `"safe"`: Expand eligible dotted keys according to the rules below.
651
+
652
+ Safe mode behavior:
653
+ - Any key containing the path separator (`.`) is considered for expansion.
654
+ - Split the key into segments at each occurrence of `.`.
655
+ - Only expand when ALL resulting segments are IdentifierSegments (§1.9) and none contain `.` after splitting.
656
+ - Keys that do not meet the expansion criteria remain as literal keys.
657
+
658
+ Deep merge semantics:
659
+ When multiple expanded keys construct overlapping object paths, the decoder MUST merge them recursively:
660
+ - Object + Object: Deep merge recursively (recurse into nested keys and apply these rules).
661
+ - Object + Non-object (array or primitive): This is a conflict. Apply conflict resolution policy.
662
+ - Array + Array or Primitive + Primitive: This is a conflict. Apply conflict resolution policy. Arrays are never merged element-wise.
663
+ - Key ordering: During expansion, newly created keys are inserted in encounter order (the order they appear in the document). When merging creates nested keys, keys from later lines are appended after existing keys at the same depth. This ensures deterministic, predictable key order in the resulting object.
664
+
665
+ Conflict resolution:
666
+ - Conflict definition: A conflict occurs when expansion requires an object at a given path but finds a non-object value (array or primitive), or vice versa. A conflict also occurs when a final leaf key already exists with a non-object value that must be overwritten.
667
+ - `strict=true` (default): Decoders MUST error on any conflict. This ensures data integrity and catches structural inconsistencies.
668
+ - `strict=false`: Last-write-wins (LWW) conflict resolution: keys appearing later in document order (encounter order during parsing) overwrite earlier values. This provides deterministic behavior for lenient parsing.
669
+
670
+ Application order: Path expansion is applied AFTER all base parsing rules (§4–12) have been applied and BEFORE the final decoded value is returned to the caller. Structural validations enumerated in §14 (strict-mode errors for array counts, indentation, etc.) operate on the pre-expanded structure and remain unaffected by expansion.
671
+
672
+ Examples:
673
+ - Input: `data.meta.items[2]: a,b` with `expandPaths="safe"` → Output: `{"data": {"meta": {"items": ["a", "b"]}}}`
674
+ - Input: `user.name: Ada` with `expandPaths="off"` → Output: `{"user.name": "Ada"}`
675
+ - Input: `a.b.c: 1` and `a.b.d: 2` and `a.e: 3` with `expandPaths="safe"` → Output: `{"a": {"b": {"c": 1, "d": 2}, "e": 3}}` (deep merge)
676
+ - Input: `a.b: 1` then `a: 2` with `expandPaths="safe"` and `strict=true` → Error: "Expansion conflict at path 'a' (object vs primitive)"
677
+ - Input: `a.b: 1` then `a: 2` with `expandPaths="safe"` and `strict=false` → Output: `{"a": 2}` (LWW)
678
+
591
679
  ### 13.1 Encoder Conformance Checklist
592
680
 
593
681
  Conforming encoders MUST:
@@ -601,6 +689,8 @@ Conforming encoders MUST:
601
689
  - [ ] Convert -0 to 0 (§2)
602
690
  - [ ] Convert NaN/±Infinity to null (§3)
603
691
  - [ ] Emit no trailing spaces or trailing newline (§12)
692
+ - [ ] When `keyFolding="safe"`, folding MUST comply with §13.4 (IdentifierSegment validation, no separator in segments, collision avoidance, no quoting required)
693
+ - [ ] When `flattenDepth` is set, folding MUST stop at the configured segment count (§13.4)
604
694
 
605
695
  ### 13.2 Decoder Conformance Checklist
606
696
 
@@ -609,9 +699,12 @@ Conforming decoders MUST:
609
699
  - [ ] Split inline arrays and tabular rows using active delimiter only (§11)
610
700
  - [ ] Unescape quoted strings with only valid escapes (§7.1)
611
701
  - [ ] Type unquoted primitives: true/false/null → booleans/null, numeric → number, else → string (§4)
612
- - [ ] Enforce strict-mode rules when strict=true (§14)
702
+ - [ ] Enforce strict-mode rules when `strict=true` (§14)
613
703
  - [ ] Accept and ignore optional # length marker (§6)
614
704
  - [ ] Preserve array order and object key order (§2)
705
+ - [ ] When `expandPaths="safe"`, expansion MUST follow §13.4 (IdentifierSegment-only segments, deep merge, conflict rules)
706
+ - [ ] When `expandPaths="safe"` with `strict=true`, MUST error on expansion conflicts per §14.5
707
+ - [ ] When `expandPaths="safe"` with `strict=false`, apply LWW conflict resolution (§13.4)
615
708
 
616
709
  ### 13.3 Validator Conformance Checklist
617
710
 
@@ -650,7 +743,17 @@ When strict mode is enabled (default), decoders MUST error on the following cond
650
743
 
651
744
  For root-form rules, including handling of empty documents, see §5.
652
745
 
653
- ### 14.5 Recommended Error Messages and Validator Diagnostics (Informative)
746
+ ### 14.5 Path Expansion Conflicts
747
+
748
+ When `expandPaths="safe"` is enabled:
749
+ - With `strict=true` (default): Decoders MUST error on any expansion conflict.
750
+ - With `strict=false`: Decoders MUST apply deterministic last-write-wins (LWW) resolution in document order. Implementations MUST resolve conflicts silently and MUST NOT emit diagnostics during normal decode operations.
751
+
752
+ See §13.4 for complete conflict definitions, deep-merge semantics, and examples.
753
+
754
+ Note (informative): Implementations MAY expose conflict diagnostics via out-of-band mechanisms (e.g., debug hooks, verbose CLI flags, or separate validation APIs), but such facilities are non-normative and MUST NOT affect default decode behavior or output.
755
+
756
+ ### 14.6 Recommended Error Messages and Validator Diagnostics (Informative)
654
757
 
655
758
  Validators SHOULD additionally report:
656
759
  - Trailing spaces, trailing newlines (encoding invariants).
@@ -972,6 +1075,74 @@ Quoted keys with arrays (keys requiring quoting per Section 7.3):
972
1075
  - id: 2
973
1076
  ```
974
1077
 
1078
+ Key folding and path expansion (v1.5+):
1079
+
1080
+ Encoding - basic folding (safe mode, depth=Infinity):
1081
+
1082
+ Input: `{"a": {"b": {"c": 1}}}`
1083
+ ```
1084
+ a.b.c: 1
1085
+ ```
1086
+
1087
+ Encoding - folding with inline array:
1088
+
1089
+ Input: `{"data": {"meta": {"items": ["x", "y"]}}}`
1090
+ ```
1091
+ data.meta.items[2]: x,y
1092
+ ```
1093
+
1094
+ Encoding - folding with tabular array:
1095
+
1096
+ Input: `{"a": {"b": {"items": [{"id": 1, "name": "A"}, {"id": 2, "name": "B"}]}}}`
1097
+ ```
1098
+ a.b.items[2]{id,name}:
1099
+ 1,A
1100
+ 2,B
1101
+ ```
1102
+
1103
+ Encoding - partial folding (flattenDepth=2):
1104
+
1105
+ Input: `{"a": {"b": {"c": {"d": 1}}}}`
1106
+ ```
1107
+ a.b:
1108
+ c:
1109
+ d: 1
1110
+ ```
1111
+
1112
+ Decoding - basic expansion (safe mode round-trip):
1113
+
1114
+ Input: `data.meta.items[2]: a,b` with options `{expandPaths: "safe"}`
1115
+
1116
+ Output: `{"data": {"meta": {"items": ["a", "b"]}}}`
1117
+
1118
+ Decoding - deep merge (multiple expanded keys):
1119
+
1120
+ Input with options `{expandPaths: "safe"}`:
1121
+ ```
1122
+ a.b.c: 1
1123
+ a.b.d: 2
1124
+ a.e: 3
1125
+ ```
1126
+ Output: `{"a": {"b": {"c": 1, "d": 2}, "e": 3}}`
1127
+
1128
+ Decoding - conflict error (strict=true, default):
1129
+
1130
+ Input with options `{expandPaths: "safe", strict: true}`:
1131
+ ```
1132
+ a.b: 1
1133
+ a: 2
1134
+ ```
1135
+ Result: Error - "Expansion conflict at path 'a' (object vs primitive)"
1136
+
1137
+ Decoding - conflict LWW (strict=false):
1138
+
1139
+ Input with options `{expandPaths: "safe", strict: false}`:
1140
+ ```
1141
+ a.b: 1
1142
+ a: 2
1143
+ ```
1144
+ Output: `{"a": 2}`
1145
+
975
1146
  ## Appendix B: Parsing Helpers (Informative)
976
1147
 
977
1148
  These sketches illustrate structure and common decoding helpers. They are informative; normative behavior is defined in Sections 4–12 and 14.
@@ -1060,6 +1231,15 @@ Note: Host-type normalization tests (e.g., BigInt, Date, Set, Map) are language-
1060
1231
 
1061
1232
  ## Appendix D: Document Changelog (Informative)
1062
1233
 
1234
+ ### v1.5 (2025-11-08)
1235
+
1236
+ - Added optional key folding for encoders: `keyFolding='safe'` mode with `flattenDepth` control (§13.4).
1237
+ - Added optional path expansion for decoders: `expandPaths='safe'` mode with conflict resolution tied to existing `strict` option (§13.4).
1238
+ - Defined safe-mode requirements for folding: IdentifierSegment validation, no path separator in segments, collision avoidance, no quoting required (§7.3, §13.4).
1239
+ - Specified deep-merge semantics for expansion: recursive merge for objects; conflict policy (error in strict mode, LWW when strict=false) for non-objects (§13.4).
1240
+ - Added strict-mode error category for path expansion conflicts (§14.5).
1241
+ - Both features default to OFF; fully backward-compatible.
1242
+
1063
1243
  ### v1.4 (2025-11-05)
1064
1244
 
1065
1245
  - Removed JavaScript-specific normalization details; replaced with language-agnostic requirements (Section 3).
@@ -1249,6 +1429,7 @@ For a detailed version history, see Appendix D.
1249
1429
 
1250
1430
  - Backward-compatible evolutions SHOULD preserve current headers, quoting rules, and indentation semantics.
1251
1431
  - Reserved/structural characters (colon, brackets, braces, hyphen) MUST retain current meanings.
1432
+ - The path separator (see §1.9) is fixed to `"."` in v1.5; future versions MAY make this configurable.
1252
1433
  - Future work (non-normative): schemas, comments/annotations, additional delimiter profiles, optional \uXXXX escapes (if added, must be precisely defined).
1253
1434
 
1254
1435
  ## 21. Intellectual Property Considerations
package/VERSIONING.md CHANGED
@@ -13,7 +13,7 @@ The TOON specification follows [Semantic Versioning](https://semver.org/) with a
13
13
  - **MAJOR version** - Incremented for breaking changes that are incompatible with previous versions
14
14
  - **MINOR version** - Incremented for backward-compatible additions, clarifications, or non-breaking changes
15
15
 
16
- **Example:** Moving from v1.3 to v1.4 means your implementation keeps working. Moving from v1.3 to v2.0 means you'll likely need to update your code.
16
+ **Example:** Moving from v1.5 to v1.6 means your implementation keeps working. Moving from v1.5 to v2.0 means you'll likely need to update your code.
17
17
 
18
18
  ## What Constitutes a Breaking Change
19
19
 
package/package.json CHANGED
@@ -1,7 +1,7 @@
1
1
  {
2
2
  "name": "@toon-format/spec",
3
3
  "type": "module",
4
- "version": "1.4.0",
4
+ "version": "1.5.0",
5
5
  "packageManager": "pnpm@10.19.0",
6
6
  "description": "Official specification for Token-Oriented Object Notation (TOON)",
7
7
  "author": "Johann Schopplich <hello@johannschopplich.com>",
package/tests/README.md CHANGED
@@ -26,7 +26,8 @@ tests/
26
26
  │ │ ├── arrays-objects.json
27
27
  │ │ ├── delimiters.json
28
28
  │ │ ├── whitespace.json
29
- │ │ └── options.json
29
+ │ │ ├── options.json
30
+ │ │ └── key-folding.json
30
31
  │ └── decode/ # Decoding tests (TOON → JSON)
31
32
  │ ├── primitives.json
32
33
  │ ├── numbers.json
@@ -39,7 +40,8 @@ tests/
39
40
  │ ├── root-form.json
40
41
  │ ├── validation-errors.json
41
42
  │ ├── indentation-errors.json
42
- └── blank-lines.json
43
+ ├── blank-lines.json
44
+ │ └── path-expansion.json
43
45
  └── README.md # This file
44
46
  ```
45
47
 
@@ -90,25 +92,31 @@ All test fixtures follow a standard JSON structure defined in [`fixtures.schema.
90
92
  {
91
93
  "delimiter": ",",
92
94
  "indent": 2,
93
- "lengthMarker": "#"
95
+ "lengthMarker": "#",
96
+ "keyFolding": "safe",
97
+ "flattenDepth": 3
94
98
  }
95
99
  ```
96
100
 
97
- - `delimiter`: `","` (comma, default), `"\t"` (tab), or `"|"` (pipe). Affects encoder output for multiline object values; decoders parse what's present
101
+ - `delimiter`: `","` (comma, default), `"\t"` (tab), or `"|"` (pipe). Affects encoder output; decoders parse the delimiter declared in array headers
98
102
  - `indent`: Number of spaces per indentation level (default: `2`)
99
103
  - `lengthMarker`: Optional. Set to `"#"` to prefix array lengths (e.g., `[#3]`). Omit this property to disable length markers
104
+ - `keyFolding`: `"off"` (default) or `"safe"`. Enables key folding to collapse single-key object chains into dotted-path notation (v1.5+)
105
+ - `flattenDepth`: Integer. Maximum depth to fold key chains when `keyFolding` is `"safe"` (default: Infinity). Values less than 2 have no practical folding effect (v1.5+)
100
106
 
101
107
  #### Decoding Options
102
108
 
103
109
  ```json
104
110
  {
105
111
  "indent": 2,
106
- "strict": true
112
+ "strict": true,
113
+ "expandPaths": "safe"
107
114
  }
108
115
  ```
109
116
 
110
- - `indent`: Expected number of spaces per level (default: `2`)
111
- - `strict`: Enable strict validation (default: `true`)
117
+ - `indent`: Expected number of spaces per indentation level (default: `2`)
118
+ - `strict`: Enable strict validation (default: `true`). When `expandPaths` is `"safe"`, strict mode controls conflict resolution: errors on conflicts when `true`, LWW when `false` (v1.5+)
119
+ - `expandPaths`: `"off"` (default) or `"safe"`. Enables path expansion to split dotted keys into nested object structures (v1.5+)
112
120
 
113
121
  ### Error Tests
114
122
 
@@ -154,6 +162,7 @@ The fixture format is language-agnostic JSON, so you can load and iterate it usi
154
162
  | `delimiters.json` | Tab and pipe delimiter options | §11 |
155
163
  | `whitespace.json` | Formatting invariants and indentation | §12 |
156
164
  | `options.json` | Length marker and delimiter option combinations | §3 |
165
+ | `key-folding.json` | Key folding with safe mode, depth control, collision avoidance | §13.4 |
157
166
 
158
167
  ### Decoding Tests (`fixtures/decode/`)
159
168
 
@@ -171,6 +180,7 @@ The fixture format is language-agnostic JSON, so you can load and iterate it usi
171
180
  | `validation-errors.json` | Syntax errors, length mismatches, malformed input | §14 |
172
181
  | `indentation-errors.json` | Strict mode indentation validation | §14.3, §12 |
173
182
  | `blank-lines.json` | Blank line handling in arrays | §14.4, §12 |
183
+ | `path-expansion.json` | Path expansion with safe mode, deep merge, strict-mode conflicts | §13.4, §14.5 |
174
184
 
175
185
  ## Validating Fixtures
176
186
 
@@ -0,0 +1,173 @@
1
+ {
2
+ "version": "1.5",
3
+ "category": "decode",
4
+ "description": "Path expansion with safe mode, deep merge, conflict resolution tied to strict mode",
5
+ "tests": [
6
+ {
7
+ "name": "expands dotted key to nested object in safe mode",
8
+ "input": "a.b.c: 1",
9
+ "expected": {
10
+ "a": {
11
+ "b": {
12
+ "c": 1
13
+ }
14
+ }
15
+ },
16
+ "options": {
17
+ "expandPaths": "safe"
18
+ },
19
+ "specSection": "13.4"
20
+ },
21
+ {
22
+ "name": "expands dotted key with inline array",
23
+ "input": "data.meta.items[2]: a,b",
24
+ "expected": {
25
+ "data": {
26
+ "meta": {
27
+ "items": ["a", "b"]
28
+ }
29
+ }
30
+ },
31
+ "options": {
32
+ "expandPaths": "safe"
33
+ },
34
+ "specSection": "13.4"
35
+ },
36
+ {
37
+ "name": "expands dotted key with tabular array",
38
+ "input": "a.b.items[2]{id,name}:\n 1,A\n 2,B",
39
+ "expected": {
40
+ "a": {
41
+ "b": {
42
+ "items": [
43
+ { "id": 1, "name": "A" },
44
+ { "id": 2, "name": "B" }
45
+ ]
46
+ }
47
+ }
48
+ },
49
+ "options": {
50
+ "expandPaths": "safe"
51
+ },
52
+ "specSection": "13.4"
53
+ },
54
+ {
55
+ "name": "preserves literal dotted keys when expansion is off",
56
+ "input": "user.name: Ada",
57
+ "expected": {
58
+ "user.name": "Ada"
59
+ },
60
+ "options": {
61
+ "expandPaths": "off"
62
+ },
63
+ "specSection": "13.4"
64
+ },
65
+ {
66
+ "name": "expands and deep-merges preserving document-order insertion",
67
+ "input": "a.b.c: 1\na.b.d: 2\na.e: 3",
68
+ "expected": {
69
+ "a": {
70
+ "b": {
71
+ "c": 1,
72
+ "d": 2
73
+ },
74
+ "e": 3
75
+ }
76
+ },
77
+ "options": {
78
+ "expandPaths": "safe"
79
+ },
80
+ "specSection": "13.4"
81
+ },
82
+ {
83
+ "name": "throws on expansion conflict (object vs primitive) when strict=true",
84
+ "input": "a.b: 1\na: 2",
85
+ "expected": null,
86
+ "shouldError": true,
87
+ "options": {
88
+ "expandPaths": "safe",
89
+ "strict": true
90
+ },
91
+ "specSection": "14.5"
92
+ },
93
+ {
94
+ "name": "throws on expansion conflict (object vs array) when strict=true",
95
+ "input": "a.b: 1\na[2]: 2,3",
96
+ "expected": null,
97
+ "shouldError": true,
98
+ "options": {
99
+ "expandPaths": "safe",
100
+ "strict": true
101
+ },
102
+ "specSection": "14.5"
103
+ },
104
+ {
105
+ "name": "applies LWW when strict=false (primitive overwrites expanded object)",
106
+ "input": "a.b: 1\na: 2",
107
+ "expected": {
108
+ "a": 2
109
+ },
110
+ "options": {
111
+ "expandPaths": "safe",
112
+ "strict": false
113
+ },
114
+ "specSection": "13.4",
115
+ "note": "Document order determines winner: later key overwrites earlier"
116
+ },
117
+ {
118
+ "name": "applies LWW when strict=false (expanded object overwrites primitive)",
119
+ "input": "a: 1\na.b: 2",
120
+ "expected": {
121
+ "a": {
122
+ "b": 2
123
+ }
124
+ },
125
+ "options": {
126
+ "expandPaths": "safe",
127
+ "strict": false
128
+ },
129
+ "specSection": "13.4",
130
+ "note": "Document order determines winner: later key overwrites earlier"
131
+ },
132
+ {
133
+ "name": "preserves quoted dotted key as literal when expandPaths=safe",
134
+ "input": "a.b: 1\n\"c.d\": 2",
135
+ "expected": {
136
+ "a": {
137
+ "b": 1
138
+ },
139
+ "c.d": 2
140
+ },
141
+ "options": {
142
+ "expandPaths": "safe"
143
+ },
144
+ "specSection": "13.4"
145
+ },
146
+ {
147
+ "name": "preserves non-IdentifierSegment keys as literals",
148
+ "input": "full-name.x: 1",
149
+ "expected": {
150
+ "full-name.x": 1
151
+ },
152
+ "options": {
153
+ "expandPaths": "safe"
154
+ },
155
+ "specSection": "13.4"
156
+ },
157
+ {
158
+ "name": "expands keys creating empty nested objects",
159
+ "input": "a.b.c:",
160
+ "expected": {
161
+ "a": {
162
+ "b": {
163
+ "c": {}
164
+ }
165
+ }
166
+ },
167
+ "options": {
168
+ "expandPaths": "safe"
169
+ },
170
+ "specSection": "13.4"
171
+ }
172
+ ]
173
+ }
@@ -0,0 +1,218 @@
1
+ {
2
+ "version": "1.5",
3
+ "category": "encode",
4
+ "description": "Key folding with safe mode, depth control, collision avoidance",
5
+ "tests": [
6
+ {
7
+ "name": "encodes folded chain to primitive (safe mode)",
8
+ "input": {
9
+ "a": {
10
+ "b": {
11
+ "c": 1
12
+ }
13
+ }
14
+ },
15
+ "expected": "a.b.c: 1",
16
+ "options": {
17
+ "keyFolding": "safe"
18
+ },
19
+ "specSection": "13.4"
20
+ },
21
+ {
22
+ "name": "encodes folded chain with inline array",
23
+ "input": {
24
+ "data": {
25
+ "meta": {
26
+ "items": ["x", "y"]
27
+ }
28
+ }
29
+ },
30
+ "expected": "data.meta.items[2]: x,y",
31
+ "options": {
32
+ "keyFolding": "safe"
33
+ },
34
+ "specSection": "13.4"
35
+ },
36
+ {
37
+ "name": "encodes folded chain with tabular array",
38
+ "input": {
39
+ "a": {
40
+ "b": {
41
+ "items": [
42
+ { "id": 1, "name": "A" },
43
+ { "id": 2, "name": "B" }
44
+ ]
45
+ }
46
+ }
47
+ },
48
+ "expected": "a.b.items[2]{id,name}:\n 1,A\n 2,B",
49
+ "options": {
50
+ "keyFolding": "safe"
51
+ },
52
+ "specSection": "13.4"
53
+ },
54
+ {
55
+ "name": "skips folding when segment requires quotes (safe mode)",
56
+ "input": {
57
+ "data": {
58
+ "full-name": {
59
+ "x": 1
60
+ }
61
+ }
62
+ },
63
+ "expected": "data:\n \"full-name\":\n x: 1",
64
+ "options": {
65
+ "keyFolding": "safe"
66
+ },
67
+ "specSection": "13.4"
68
+ },
69
+ {
70
+ "name": "skips folding on sibling literal-key collision (safe mode)",
71
+ "input": {
72
+ "data": {
73
+ "meta": {
74
+ "items": [1, 2]
75
+ }
76
+ },
77
+ "data.meta.items": "literal"
78
+ },
79
+ "expected": "data:\n meta:\n items[2]: 1,2\ndata.meta.items: literal",
80
+ "options": {
81
+ "keyFolding": "safe"
82
+ },
83
+ "specSection": "13.4",
84
+ "note": "Collision avoidance: folding would create duplicate key"
85
+ },
86
+ {
87
+ "name": "encodes partial folding with flattenDepth=2",
88
+ "input": {
89
+ "a": {
90
+ "b": {
91
+ "c": {
92
+ "d": 1
93
+ }
94
+ }
95
+ }
96
+ },
97
+ "expected": "a.b:\n c:\n d: 1",
98
+ "options": {
99
+ "keyFolding": "safe",
100
+ "flattenDepth": 2
101
+ },
102
+ "specSection": "13.4"
103
+ },
104
+ {
105
+ "name": "encodes full chain with flattenDepth=Infinity (default)",
106
+ "input": {
107
+ "a": {
108
+ "b": {
109
+ "c": {
110
+ "d": 1
111
+ }
112
+ }
113
+ }
114
+ },
115
+ "expected": "a.b.c.d: 1",
116
+ "options": {
117
+ "keyFolding": "safe"
118
+ },
119
+ "specSection": "13.4"
120
+ },
121
+ {
122
+ "name": "encodes standard nesting with flattenDepth=0 (no folding)",
123
+ "input": {
124
+ "a": {
125
+ "b": {
126
+ "c": 1
127
+ }
128
+ }
129
+ },
130
+ "expected": "a:\n b:\n c: 1",
131
+ "options": {
132
+ "keyFolding": "safe",
133
+ "flattenDepth": 0
134
+ },
135
+ "specSection": "13.4",
136
+ "note": "flattenDepth=0 disables all folding"
137
+ },
138
+ {
139
+ "name": "encodes standard nesting with flattenDepth=1 (no practical effect)",
140
+ "input": {
141
+ "a": {
142
+ "b": {
143
+ "c": 1
144
+ }
145
+ }
146
+ },
147
+ "expected": "a:\n b:\n c: 1",
148
+ "options": {
149
+ "keyFolding": "safe",
150
+ "flattenDepth": 1
151
+ },
152
+ "specSection": "13.4",
153
+ "note": "flattenDepth=1 has no practical folding effect (requires at least 2 segments)"
154
+ },
155
+ {
156
+ "name": "encodes standard nesting with keyFolding=off (baseline)",
157
+ "input": {
158
+ "a": {
159
+ "b": {
160
+ "c": 1
161
+ }
162
+ }
163
+ },
164
+ "expected": "a:\n b:\n c: 1",
165
+ "options": {
166
+ "keyFolding": "off"
167
+ },
168
+ "specSection": "13.4"
169
+ },
170
+ {
171
+ "name": "encodes folded chain ending with empty object",
172
+ "input": {
173
+ "a": {
174
+ "b": {
175
+ "c": {}
176
+ }
177
+ }
178
+ },
179
+ "expected": "a.b.c:",
180
+ "options": {
181
+ "keyFolding": "safe"
182
+ },
183
+ "specSection": "13.4"
184
+ },
185
+ {
186
+ "name": "stops folding at array boundary (not single-key object)",
187
+ "input": {
188
+ "a": {
189
+ "b": [1, 2]
190
+ }
191
+ },
192
+ "expected": "a.b[2]: 1,2",
193
+ "options": {
194
+ "keyFolding": "safe"
195
+ },
196
+ "specSection": "13.4"
197
+ },
198
+ {
199
+ "name": "encodes partial fold preserving sibling field order",
200
+ "input": {
201
+ "folded": {
202
+ "path": {
203
+ "value": 1
204
+ }
205
+ },
206
+ "normal": 2,
207
+ "nested": {
208
+ "x": 3
209
+ }
210
+ },
211
+ "expected": "folded.path.value: 1\nnormal: 2\nnested:\n x: 3",
212
+ "options": {
213
+ "keyFolding": "safe"
214
+ },
215
+ "specSection": "13.4"
216
+ }
217
+ ]
218
+ }
@@ -76,6 +76,23 @@
76
76
  "type": "boolean",
77
77
  "description": "Enable strict validation (decode only)",
78
78
  "default": true
79
+ },
80
+ "keyFolding": {
81
+ "type": "string",
82
+ "enum": ["off", "safe"],
83
+ "description": "Key folding mode for encoders (v1.5+): 'off' (default) or 'safe' (encode only)",
84
+ "default": "off"
85
+ },
86
+ "flattenDepth": {
87
+ "type": "integer",
88
+ "description": "Maximum depth to fold key chains when keyFolding is 'safe' (v1.5+). Values less than 2 have no practical folding effect (encode only)",
89
+ "minimum": 0
90
+ },
91
+ "expandPaths": {
92
+ "type": "string",
93
+ "enum": ["off", "safe"],
94
+ "description": "Path expansion mode for decoders (v1.5+): 'off' (default) or 'safe' (decode only)",
95
+ "default": "off"
79
96
  }
80
97
  },
81
98
  "additionalProperties": false