gcf-python 2.1.1__tar.gz → 2.2.1__tar.gz

{gcf_python-2.1.1 → gcf_python-2.2.1}/CHANGELOG.md

@@ -1,5 +1,29 @@
 # Changelog
 
+## v2.2.1 (2026-06-23)
+
+### Flatten Opt-Out
+
+- Added `GenericOptions` dataclass with `no_flatten` field to disable nested object flattening
+- `encode_generic(data, GenericOptions(no_flatten=True))` produces attachment syntax instead of path columns
+- Backward compatible: `encode_generic(data)` behavior unchanged (flatten on by default)
+- Fixed: field names containing `>` no longer appear as tabular columns (spec rule 7.4.6.1.4)
+- Fixed: field names containing `>` no longer eligible for flattening analysis
+- Fixed: decoder no longer treats literal `>` in key names as a path separator
+- Fixed: decoder accepts orphan attachments (fields excluded from column list)
+- 12 targeted edge case tests for `>` in field names
+
+## v2.2.0 (2026-06-22)
+
+### Spec v3.2: Nested Object Flattening
+
+- Encoder automatically flattens fixed-shape nested objects into `>` path column names (e.g., `"customer>name"` instead of `^` + `.customer {}` attachment)
+- Decoder reconstructs nested objects from `>` path columns
+- 20-48% fewer tokens on deeply nested API data (Jira, Stripe, K8s, calendar events)
+- 100% comprehension on every frontier model (validated across 9 models, 7 providers)
+- Zero regression on lossless round-trips (230 tests, conformance + property-based)
+- Falls back to attachment mechanism for: variable-length arrays, objects with different keys across rows, objects with `>` in key names, empty nested objects
+
 ## v2.1.0 (2026-06-14)
 
 ### Spec v3.1

{gcf_python-2.1.1 → gcf_python-2.2.1}/PKG-INFO

@@ -1,7 +1,7 @@
 Metadata-Version: 2.4
 Name: gcf-python
-Version: 2.1.1
-Summary: Drop-in JSON replacement for AI pipelines. 79% fewer tokens. 90.7% comprehension across 10 models. Zero dependencies.
+Version: 2.2.1
+Summary: The AI-native wire format for structured data. 50-92% fewer tokens than JSON. 100% comprehension on every frontier model. Zero dependencies.
 Project-URL: Homepage, https://github.com/blackwell-systems/gcf-python
 Project-URL: Documentation, https://blackwell-systems.github.io/gcf/
 Project-URL: Specification, https://github.com/blackwell-systems/gcf
@@ -32,7 +32,7 @@ Description-Content-Type: text/markdown
 
 Python implementation of [GCF](https://gcformat.com/) — the most token-efficient wire format for LLMs. A drop-in alternative to JSON and TOON for any structured data.
 
-**100% comprehension on every frontier model tested. 25.5% fewer tokens than TOON, 53% fewer than JSON across 15 datasets. 90.7% on structurally complex code graphs (vs TOON 68.5%, JSON 53.6%). 1,700+ LLM evaluations. Zero training.**
+**100% comprehension on every frontier model tested. 29% fewer tokens than TOON, 56% fewer than JSON across 16 datasets. 91.2% on structurally complex code graphs (vs TOON 68.2%, JSON 53.4%). 2,400+ LLM evaluations. Zero training.**
 
 Docs: [gcformat.com](https://gcformat.com/) · [Playground](https://gcformat.com/playground.html) · [GCF vs TOON](https://gcformat.com/guide/vs-toon.html)
 
@@ -206,16 +206,16 @@ Works on dicts, lists, and primitives. Lists of uniform dicts get tabular rows.
 
 ## Benchmarks
 
-1,700+ LLM evaluations across 10 models, 3 providers, and 51 independent test runs.
+2,400+ LLM evaluations across 10 models, 3 providers, and 51 independent test runs.
 
 | | GCF | TOON | JSON |
 |---|---|---|---|
-| **Comprehension** (23 runs, 10 models) | **90.7%** | 68.5% | 53.6% |
+| **Comprehension** (23 runs, 10 models) | **91.2%** | 68.2% | 53.4% |
 | **Generation** (28 runs, 9 models) | **5/5** | 1.0/5 | 5.0/5 |
 | **Input tokens** (500 symbols) | **11,090** | 16,378 | 53,341 |
 | **Output tokens** (100 symbols) | **5,976** | 8,937 | 16,121 |
 
-GCF wins 13/15 datasets on the expanded [token efficiency benchmark](https://github.com/blackwell-systems/toon/tree/gcf-comparison). Full results: [gcformat.com/guide/benchmarks](https://gcformat.com/guide/benchmarks.html)
+GCF wins 15/16 datasets on the expanded [token efficiency benchmark](https://github.com/blackwell-systems/toon/tree/gcf-comparison). Full results: [gcformat.com/guide/benchmarks](https://gcformat.com/guide/benchmarks.html)
 
 ## Implementations
 
@@ -234,9 +234,9 @@ GCF wins 13/15 datasets on the expanded [token efficiency benchmark](https://git
 | n8n | `npm install n8n-nodes-gcf` | [gcf-n8n-nodes](https://github.com/blackwell-systems/gcf-n8n-nodes) (workflow encode/decode) |
 | Tree-sitter | `npm install tree-sitter-gcf` | [tree-sitter-gcf](https://github.com/blackwell-systems/tree-sitter-gcf) |
 
-Zero runtime dependencies. MIT licensed. All implementations support both generic profile (`encodeGeneric`) and graph profile (`encode`). CLI included in all 6 languages.
+**Zero runtime dependencies. Permanently.** All six implementations depend only on their language's standard library. No transitive dependencies. No supply chain risk. This is a permanent commitment: GCF will never take on external runtime dependencies. MIT licensed. All implementations support both generic profile (`encodeGeneric`) and graph profile (`encode`). CLI included in all 6 languages.
 
-**Specification:** [SPEC v3.1 Stable](https://github.com/blackwell-systems/gcf/blob/main/SPEC.md) with 157 conformance fixtures, 33,000,000,000+ lossless round-trips verified across 5 formats and 6 languages. All implementations at v2.1.0+ (Go v1.2.0). Cross-language 6x6 matrix verified.
+**Specification:** [SPEC v3.2 Stable](https://github.com/blackwell-systems/gcf/blob/main/SPEC.md) with 173 conformance fixtures, 43,000,000,000+ lossless round-trips verified across 5 formats and 6 languages. All implementations at v2.2.0+ (Go v1.3.0). Cross-language 6x6 matrix verified.
 
 ## License
 

{gcf_python-2.1.1 → gcf_python-2.2.1}/README.md

@@ -7,7 +7,7 @@
 
 Python implementation of [GCF](https://gcformat.com/) — the most token-efficient wire format for LLMs. A drop-in alternative to JSON and TOON for any structured data.
 
-**100% comprehension on every frontier model tested. 25.5% fewer tokens than TOON, 53% fewer than JSON across 15 datasets. 90.7% on structurally complex code graphs (vs TOON 68.5%, JSON 53.6%). 1,700+ LLM evaluations. Zero training.**
+**100% comprehension on every frontier model tested. 29% fewer tokens than TOON, 56% fewer than JSON across 16 datasets. 91.2% on structurally complex code graphs (vs TOON 68.2%, JSON 53.4%). 2,400+ LLM evaluations. Zero training.**
 
 Docs: [gcformat.com](https://gcformat.com/) · [Playground](https://gcformat.com/playground.html) · [GCF vs TOON](https://gcformat.com/guide/vs-toon.html)
 
@@ -181,16 +181,16 @@ Works on dicts, lists, and primitives. Lists of uniform dicts get tabular rows.
 
 ## Benchmarks
 
-1,700+ LLM evaluations across 10 models, 3 providers, and 51 independent test runs.
+2,400+ LLM evaluations across 10 models, 3 providers, and 51 independent test runs.
 
 | | GCF | TOON | JSON |
 |---|---|---|---|
-| **Comprehension** (23 runs, 10 models) | **90.7%** | 68.5% | 53.6% |
+| **Comprehension** (23 runs, 10 models) | **91.2%** | 68.2% | 53.4% |
 | **Generation** (28 runs, 9 models) | **5/5** | 1.0/5 | 5.0/5 |
 | **Input tokens** (500 symbols) | **11,090** | 16,378 | 53,341 |
 | **Output tokens** (100 symbols) | **5,976** | 8,937 | 16,121 |
 
-GCF wins 13/15 datasets on the expanded [token efficiency benchmark](https://github.com/blackwell-systems/toon/tree/gcf-comparison). Full results: [gcformat.com/guide/benchmarks](https://gcformat.com/guide/benchmarks.html)
+GCF wins 15/16 datasets on the expanded [token efficiency benchmark](https://github.com/blackwell-systems/toon/tree/gcf-comparison). Full results: [gcformat.com/guide/benchmarks](https://gcformat.com/guide/benchmarks.html)
 
 ## Implementations
 
@@ -209,9 +209,9 @@ GCF wins 13/15 datasets on the expanded [token efficiency benchmark](https://git
 | n8n | `npm install n8n-nodes-gcf` | [gcf-n8n-nodes](https://github.com/blackwell-systems/gcf-n8n-nodes) (workflow encode/decode) |
 | Tree-sitter | `npm install tree-sitter-gcf` | [tree-sitter-gcf](https://github.com/blackwell-systems/tree-sitter-gcf) |
 
-Zero runtime dependencies. MIT licensed. All implementations support both generic profile (`encodeGeneric`) and graph profile (`encode`). CLI included in all 6 languages.
+**Zero runtime dependencies. Permanently.** All six implementations depend only on their language's standard library. No transitive dependencies. No supply chain risk. This is a permanent commitment: GCF will never take on external runtime dependencies. MIT licensed. All implementations support both generic profile (`encodeGeneric`) and graph profile (`encode`). CLI included in all 6 languages.
 
-**Specification:** [SPEC v3.1 Stable](https://github.com/blackwell-systems/gcf/blob/main/SPEC.md) with 157 conformance fixtures, 33,000,000,000+ lossless round-trips verified across 5 formats and 6 languages. All implementations at v2.1.0+ (Go v1.2.0). Cross-language 6x6 matrix verified.
+**Specification:** [SPEC v3.2 Stable](https://github.com/blackwell-systems/gcf/blob/main/SPEC.md) with 173 conformance fixtures, 43,000,000,000+ lossless round-trips verified across 5 formats and 6 languages. All implementations at v2.2.0+ (Go v1.3.0). Cross-language 6x6 matrix verified.
 
 ## License
 

{gcf_python-2.1.1 → gcf_python-2.2.1}/pyproject.toml

@@ -4,8 +4,8 @@ build-backend = "hatchling.build"
 
 [project]
 name = "gcf-python"
-version = "2.1.1"
-description = "Drop-in JSON replacement for AI pipelines. 79% fewer tokens. 90.7% comprehension across 10 models. Zero dependencies."
+version = "2.2.1"
+description = "The AI-native wire format for structured data. 50-92% fewer tokens than JSON. 100% comprehension on every frontier model. Zero dependencies."
 readme = "README.md"
 license = {text = "MIT"}
 requires-python = ">=3.9"

{gcf_python-2.1.1 → gcf_python-2.2.1}/src/gcf/__init__.py

@@ -38,7 +38,7 @@ from .constants import KIND_ABBREV, KIND_EXPAND
 from .decode import DecodeError, decode
 from .delta import encode_delta
 from .encode import encode
-from .generic import encode_generic
+from .generic import encode_generic, GenericOptions
 from .session import Session, encode_with_session
 from .decode_generic import decode_generic
 from .stream import StreamEncoder
@@ -62,6 +62,7 @@ __all__ = [
     "encode",
     "encode_delta",
     "encode_generic",
+    "GenericOptions",
     "encode_with_session",
 ]
 

{gcf_python-2.1.1 → gcf_python-2.2.1}/src/gcf/decode_generic.py

@@ -360,9 +360,66 @@ def _parse_attachment(
         arr, consumed = _parse_array_from_header(lines, line_idx, depth, after_name)
         return name, arr, consumed, None
 
+    # Scalar: =value (field names containing ">" excluded from tabular columns).
+    if after_name.startswith("="):
+        val_str = after_name[1:]
+        parsed = parse_scalar(val_str, tabular_context=True)
+        if parsed is MISSING:
+            return name, None, 1, None
+        return name, parsed, 1, None
+
     raise ValueError(f"invalid attachment form: {after_name}")
 
 
+def _unflatten_paths(
+    path_columns: dict[str, list[str]],
+    flat_values: dict[str, Any],
+    flat_absent: set[str],
+) -> dict[str, Any]:
+    """Reconstruct nested objects from flat > path columns."""
+    groups: dict[str, list[str]] = {}
+    group_order: list[str] = []
+    for field_name, paths in path_columns.items():
+        if not paths:
+            continue
+        top = paths[0]
+        if top not in groups:
+            groups[top] = []
+            group_order.append(top)
+        groups[top].append(field_name)
+
+    result: dict[str, Any] = {}
+
+    for top in group_order:
+        field_names = groups[top]
+        all_absent = all(f in flat_absent for f in field_names)
+        all_null = all(
+            (f not in flat_absent and flat_values.get(f) is None)
+            for f in field_names
+        )
+
+        if all_absent:
+            continue
+        if all_null:
+            result[top] = None
+            continue
+
+        for field_name in field_names:
+            if field_name in flat_absent:
+                continue
+            paths = path_columns[field_name]
+            val = flat_values.get(field_name)
+
+            current = result
+            for k in paths[:-1]:
+                if k not in current:
+                    current[k] = {}
+                current = current[k]
+            current[paths[-1]] = val
+
+    return result
+
+
 def _parse_tabular_body(
     lines: list[str], start: int, depth: int, fields: list[str], expected_count: int
 ) -> tuple[list[Any], int]:
@@ -370,6 +427,16 @@ def _parse_tabular_body(
     rows: list[Any] = []
     i = start
 
+    # Detect path columns: fields containing ">".
+    path_column_map: dict[str, list[str]] = {}
+    for f in fields:
+        if ">" in f:
+            parts = f.split(">")
+            # Only treat as a path column if all segments are non-empty.
+            # A literal key like ">" would split into ["", ""].
+            if all(p for p in parts):
+                path_column_map[f] = parts
+
     # Track inline schemas and shared array schemas.
     inline_schemas: dict[str, list[str]] = {}
     shared_array_schemas: dict[str, list[str]] = {}
@@ -409,9 +476,22 @@ def _parse_tabular_body(
         inline_att_order: list[str] = []
         missing_fields: set[str] = set()
 
+        # Collect path column values for unflattening.
+        flat_values: dict[str, Any] = {}
+        flat_absent: set[str] = set()
+
         for j, f in enumerate(fields):
             cell_val = vals[j]
 
+            # Path columns: store values for later unflattening.
+            if f in path_column_map:
+                parsed = parse_scalar(cell_val, tabular_context=True)
+                if parsed is MISSING:
+                    flat_absent.add(f)
+                else:
+                    flat_values[f] = parsed
+                continue
+
             # Check for ^{fields} inline schema declaration.
             if cell_val.startswith("^{") and cell_val.endswith("}"):
                 schema_str = cell_val[1:]
@@ -438,10 +518,10 @@ def _parse_tabular_body(
         all_att_fields = traditional_att_fields + inline_att_fields
         attachment_values: dict[str, Any] = {}
 
-        if row_has_id and all_att_fields:
+        if row_has_id:
             inline_idx = 0
 
-            while i < len(lines) and len(attachment_values) < len(all_att_fields):
+            while i < len(lines):
                 a_line = lines[i]
                 a_content: str | None = None
                 if depth == 0 or a_line.startswith(ind):
@@ -533,13 +613,6 @@ def _parse_tabular_body(
                     if extra_name in attachment_values:
                         raise ValueError(f"duplicate_attachment: {extra_name}")
 
-        if not row_has_id or not all_att_fields:
-            att_indent = ind + "  "
-            if i < len(lines) and lines[i].startswith(att_indent):
-                peek = lines[i][len(att_indent):]
-                if peek.startswith("."):
-                    raise ValueError(f"orphan_attachment: {peek}")
-
         row: dict[str, Any] = {}
         for f in fields:
             if f in missing_fields:
@@ -548,6 +621,15 @@ def _parse_tabular_body(
                 row[f] = cell_values[f]
             elif f in attachment_values:
                 row[f] = attachment_values[f]
+        # Also add any orphan attachment values (fields excluded from column list, e.g. ">" fields).
+        for k, v in attachment_values.items():
+            if k not in row:
+                row[k] = v
+        # Unflatten path columns into nested objects.
+        if path_column_map:
+            nested = _unflatten_paths(path_column_map, flat_values, flat_absent)
+            row.update(nested)
+
         rows.append(row)
 
         if expected_count >= 0 and len(rows) >= expected_count:

{gcf_python-2.1.1 → gcf_python-2.2.1}/src/gcf/generic.py

@@ -2,42 +2,55 @@
 
 from __future__ import annotations
 
+from dataclasses import dataclass
 from typing import Any
 
 from .scalar import format_scalar, format_key
 
 
-def encode_generic(data: Any) -> str:
+@dataclass
+class GenericOptions:
+    """Options for controlling generic encoding behavior."""
+    no_flatten: bool = False
+    """When True, disables promotion of fixed-shape nested objects to path
+    columns (e.g. "customer>name"). Nested objects use attachment syntax
+    instead. Set when targeting open-weight models that show lower
+    comprehension on flattened encoding."""
+
+
+def encode_generic(data: Any, opts: GenericOptions | None = None) -> str:
+    if opts is None:
+        opts = GenericOptions()
     out: list[str] = ["GCF profile=generic"]
-    _encode_root_value(data, out)
+    _encode_root_value(data, out, opts)
     return "\n".join(out) + "\n"
 
 
-def _encode_root_value(v: Any, out: list[str]) -> None:
+def _encode_root_value(v: Any, out: list[str], opts: GenericOptions) -> None:
     if v is None:
         out.append("=-")
     elif isinstance(v, dict):
-        _encode_object(v, out, 0)
+        _encode_object(v, out, 0, opts)
     elif isinstance(v, list):
-        _encode_root_array(v, out)
+        _encode_root_array(v, out, opts)
     else:
         out.append(f"={format_scalar(v)}")
 
 
-def _encode_object(d: dict, out: list[str], depth: int) -> None:
+def _encode_object(d: dict, out: list[str], depth: int, opts: GenericOptions) -> None:
     prefix = _indent(depth)
     for key, value in d.items():
         fk = format_key(key)
         if isinstance(value, dict):
             out.append(f"{prefix}## {fk}")
-            _encode_object(value, out, depth + 1)
+            _encode_object(value, out, depth + 1, opts)
         elif isinstance(value, list):
-            _encode_named_array(fk, value, out, depth)
+            _encode_named_array(fk, value, out, depth, opts)
         else:
             out.append(f"{prefix}{fk}={format_scalar(value)}")
 
 
-def _encode_root_array(arr: list, out: list[str]) -> None:
+def _encode_root_array(arr: list, out: list[str], opts: GenericOptions) -> None:
     if not arr:
         out.append("## [0]")
         return
@@ -47,12 +60,12 @@ def _encode_root_array(arr: list, out: list[str]) -> None:
         return
     fields = _tabular_fields(arr)
     if fields is not None:
-        _encode_tabular("## ", arr, fields, out, 0)
+        _encode_tabular("## ", arr, fields, out, 0, opts)
         return
-    _encode_expanded("## ", arr, out, 0)
+    _encode_expanded("## ", arr, out, 0, opts)
 
 
-def _encode_named_array(name: str, arr: list, out: list[str], depth: int) -> None:
+def _encode_named_array(name: str, arr: list, out: list[str], depth: int, opts: GenericOptions) -> None:
     prefix = _indent(depth)
     if not arr:
         out.append(f"{prefix}## {name} [0]")
@@ -63,9 +76,9 @@ def _encode_named_array(name: str, arr: list, out: list[str], depth: int) -> Non
         return
     fields = _tabular_fields(arr)
     if fields is not None:
-        _encode_tabular(f"{prefix}## {name} ", arr, fields, out, depth)
+        _encode_tabular(f"{prefix}## {name} ", arr, fields, out, depth, opts)
         return
-    _encode_expanded(f"{prefix}## {name} ", arr, out, depth)
+    _encode_expanded(f"{prefix}## {name} ", arr, out, depth, opts)
 
 
 def _tabular_fields(arr: list) -> list[str] | None:
@@ -145,15 +158,152 @@ def _shared_array_schema(arr: list[dict], field_name: str) -> list[str] | None:
     return canonical_fields
 
 
+# ── Nested object flattening (v3.2) ──────────────────────────────────────
+
+
+def _analyze_flattenable(
+    arr: list[dict], field_name: str, parent_path: str
+) -> list[dict] | None:
+    """Analyze whether a field can be flattened. Returns list of leaf descriptors or None."""
+    # Field names containing ">" cannot be flattened (would create ambiguous paths).
+    if ">" in field_name:
+        return None
+    canonical_shape: dict[str, str] | None = None  # key -> "scalar" | "nested"
+
+    for item in arr:
+        if field_name not in item or item[field_name] is None:
+            continue
+        v = item[field_name]
+        if not isinstance(v, dict):
+            return None
+        if isinstance(v, list):
+            return None
+
+        keys = list(v.keys())
+
+        if canonical_shape is None:
+            canonical_shape = {}
+            for k in keys:
+                if ">" in k:
+                    return None
+                val = v[k]
+                if isinstance(val, list):
+                    return None
+                elif isinstance(val, dict):
+                    canonical_shape[k] = "nested"
+                else:
+                    canonical_shape[k] = "scalar"
+        else:
+            if len(keys) != len(canonical_shape):
+                return None
+            for k in keys:
+                if k not in canonical_shape:
+                    return None
+                val = v[k]
+                expected = canonical_shape[k]
+                if expected == "scalar":
+                    if isinstance(val, (dict, list)):
+                        return None
+                elif expected == "nested":
+                    if isinstance(val, list):
+                        return None
+                    if val is not None and not isinstance(val, dict):
+                        return None
+
+    if canonical_shape is None:
+        return None
+
+    current_path = f"{parent_path}>{field_name}" if parent_path else field_name
+    parent_keys = parent_path.split(">") + [field_name] if parent_path else [field_name]
+
+    leaves: list[dict] = []
+    for k in canonical_shape:
+        if canonical_shape[k] == "scalar":
+            leaves.append({"path": f"{current_path}>{k}", "keys": parent_keys + [k]})
+        else:
+            sub_arr = []
+            for item in arr:
+                if field_name not in item or item[field_name] is None:
+                    sub_arr.append({})
+                else:
+                    sub_arr.append(item[field_name])
+            sub_leaves = _analyze_flattenable(sub_arr, k, current_path)
+            if sub_leaves is None or len(sub_leaves) == 0:
+                return None
+            leaves.extend(sub_leaves)
+
+    # Guard: reject if any row has non-null object with all-null leaves.
+    if leaves:
+        for item in arr:
+            if field_name not in item or item[field_name] is None:
+                continue
+            all_null = all(
+                _resolve_key_chain(item, leaf["keys"])[0] is None
+                and _resolve_key_chain(item, leaf["keys"])[1]
+                for leaf in leaves
+            )
+            if all_null:
+                return None
+
+    return leaves
+
+
+def _resolve_key_chain(item: Any, keys: list[str]) -> tuple[Any, bool]:
+    """Traverse an object by key chain. Returns (value, exists)."""
+    if not keys or not isinstance(item, dict):
+        return None, False
+    if keys[0] not in item:
+        return None, False
+    current = item[keys[0]]
+    if current is None:
+        return None, True
+    for k in keys[1:]:
+        if not isinstance(current, dict) or k not in current:
+            return None, False
+        current = current[k]
+    return current, True
+
+
 def _encode_tabular(
-    header_prefix: str, arr: list[dict], fields: list[str], out: list[str], depth: int
+    header_prefix: str, arr: list[dict], fields: list[str], out: list[str], depth: int, opts: GenericOptions
 ) -> None:
     prefix = _indent(depth)
 
-    # Pre-compute inline schemas and shared array schemas.
+    # Phase 0: Analyze fields for flattening.
+    flatten_map: dict[str, list[dict]] = {}
+    if not opts.no_flatten:
+        for f in fields:
+            leaves = _analyze_flattenable(arr, f, "")
+            if leaves and len(leaves) > 0:
+                flatten_map[f] = leaves
+
+    # Fields whose names contain ">" must not appear as tabular columns
+    # because the decoder would interpret them as flattened path columns.
+    # Track them for per-row attachment emission (spec rule 7.4.6.1.4).
+    gt_fields = {f for f in fields if f not in flatten_map and ">" in f}
+
+    # Build expanded column list.
+    columns: list[dict] = []
+    for f in fields:
+        if f in gt_fields:
+            continue
+        if f in flatten_map:
+            for leaf in flatten_map[f]:
+                columns.append({"header": format_key(leaf["path"]), "type": "flat", "field": f, "keys": leaf["keys"]})
+        else:
+            columns.append({"header": format_key(f), "type": "original", "field": f, "keys": []})
+
+    # If all fields were excluded (all contain ">"), fall back to expanded.
+    if not columns:
+        _encode_expanded(header_prefix, arr, out, depth, opts)
+        return
+
+    # Pre-compute inline schemas and shared array schemas (skip flattened fields).
     inline_schemas: dict[str, list[str]] = {}
     shared_arr_schemas: dict[str, list[str]] = {}
     for f in fields:
+        if f in flatten_map:
+            continue
         ifs = _inline_schema_fields(arr, f)
         if ifs is not None:
             inline_schemas[f] = ifs
@@ -161,15 +311,34 @@ def _encode_tabular(
         if sas is not None:
             shared_arr_schemas[f] = sas
 
-    fmt_fields = ",".join(format_key(f) for f in fields)
-    out.append(f"{header_prefix}[{len(arr)}]{{{fmt_fields}}}")
+    header_fields = ",".join(col["header"] for col in columns)
+    out.append(f"{header_prefix}[{len(arr)}]{{{header_fields}}}")
 
     for i, item in enumerate(arr):
         cells: list[str] = []
-        attachments: list[tuple[str, Any, bool, list[str] | None]] = []  # (name, value, inline, inline_fields)
+        attachments: list[tuple[str, Any, bool, list[str] | None]] = []
         row_has_attachment = False
 
-        for f in fields:
+        for col in columns:
+            if col["type"] == "flat":
+                keys = col["keys"]
+                if keys[0] not in item:
+                    cells.append("~")
+                else:
+                    top_val = item[keys[0]]
+                    if top_val is None:
+                        cells.append("-")
+                    else:
+                        val, exists = _resolve_key_chain(item, keys)
+                        if not exists:
+                            cells.append("~")
+                        elif val is None:
+                            cells.append("-")
+                        else:
+                            cells.append(format_scalar(val, "|"))
+                continue
+
+            f = col["field"]
             if f not in item:
                 cells.append("~")
                 continue
@@ -193,6 +362,15 @@ def _encode_tabular(
             else:
                 cells.append(format_scalar(v, "|"))
 
+        # Emit fields with ">" in their names as per-row attachments.
+        for f in fields:
+            if f not in gt_fields:
+                continue
+            if f not in item:
+                continue
+            row_has_attachment = True
+            attachments.append((f, item[f], False, None))
+
         row = "|".join(cells)
         if row_has_attachment:
             out.append(f"{prefix}@{i} {row}")
@@ -211,17 +389,25 @@ def _encode_tabular(
             elif isinstance(att_val, list):
                 sas = shared_arr_schemas.get(att_name)
                 if sas and i > 0:
-                    _encode_attachment_array_shared(prefix, fk, att_val, out, depth + 2, sas)
+                    _encode_attachment_array_shared(prefix, fk, att_val, out, depth + 2, sas, opts)
                 else:
-                    _encode_attachment_array(prefix, fk, att_val, out, depth + 2)
+                    _encode_attachment_array(prefix, fk, att_val, out, depth + 2, opts)
             elif isinstance(att_val, dict):
                 out.append(f"{prefix}.{fk} {{}}")
-                _encode_object(att_val, out, depth + 2)
+                _encode_object(att_val, out, depth + 2, opts)
+            else:
+                # Scalar attachment (e.g. field names containing ">").
+                if att_val is None:
+                    out.append(f"{prefix}.{fk} =-")
+                else:
+                    out.append(f"{prefix}.{fk} ={format_scalar(att_val)}")
 
 
 def _encode_attachment_array(
-    att_prefix: str, fk: str, arr: list, out: list[str], depth: int
+    att_prefix: str, fk: str, arr: list, out: list[str], depth: int, opts: GenericOptions | None = None
 ) -> None:
+    if opts is None:
+        opts = GenericOptions()
     if not arr:
         out.append(f"{att_prefix}.{fk} [0]")
     elif _all_primitives(arr):
@@ -230,13 +416,13 @@ def _encode_attachment_array(
     else:
         fields = _tabular_fields(arr)
         if fields is not None:
-            _encode_tabular(f"{att_prefix}.{fk} ", arr, fields, out, depth)
+            _encode_tabular(f"{att_prefix}.{fk} ", arr, fields, out, depth, opts)
         else:
-            _encode_expanded(f"{att_prefix}.{fk} ", arr, out, depth)
+            _encode_expanded(f"{att_prefix}.{fk} ", arr, out, depth, opts)
 
 
 def _encode_attachment_array_shared(
-    att_prefix: str, fk: str, arr: list, out: list[str], depth: int, shared_fields: list[str]
+    att_prefix: str, fk: str, arr: list, out: list[str], depth: int, shared_fields: list[str], opts: GenericOptions | None = None
 ) -> None:
     if not arr:
         out.append(f"{att_prefix}.{fk} [0]")
@@ -263,25 +449,29 @@ def _encode_attachment_array_shared(
             out.append(f"{prefix}{'|'.join(cells)}")
     else:
         # Fields don't match: fall back to full encoding.
-        _encode_attachment_array(att_prefix, fk, arr, out, depth)
+        _encode_attachment_array(att_prefix, fk, arr, out, depth, opts)
 
 
-def _encode_expanded(header_prefix: str, arr: list, out: list[str], depth: int) -> None:
+def _encode_expanded(header_prefix: str, arr: list, out: list[str], depth: int, opts: GenericOptions | None = None) -> None:
+    if opts is None:
+        opts = GenericOptions()
     prefix = _indent(depth)
     out.append(f"{header_prefix}[{len(arr)}]")
     for i, item in enumerate(arr):
         if isinstance(item, dict):
             out.append(f"{prefix}@{i} {{}}")
-            _encode_object(item, out, depth + 1)
+            _encode_object(item, out, depth + 1, opts)
         elif isinstance(item, list):
-            _encode_expanded_array_item(prefix, i, item, out, depth)
+            _encode_expanded_array_item(prefix, i, item, out, depth, opts)
         else:
             out.append(f"{prefix}@{i} ={format_scalar(item)}")
 
 
 def _encode_expanded_array_item(
-    prefix: str, idx: int, arr: list, out: list[str], depth: int
+    prefix: str, idx: int, arr: list, out: list[str], depth: int, opts: GenericOptions | None = None
 ) -> None:
+    if opts is None:
+        opts = GenericOptions()
     if not arr:
         out.append(f"{prefix}@{idx} [0]")
     elif _all_primitives(arr):
@@ -290,9 +480,9 @@ def _encode_expanded_array_item(
     else:
         fields = _tabular_fields(arr)
         if fields is not None:
-            _encode_tabular(f"{prefix}@{idx} ", arr, fields, out, depth + 1)
+            _encode_tabular(f"{prefix}@{idx} ", arr, fields, out, depth + 1, opts)
         else:
-            _encode_expanded(f"{prefix}@{idx} ", arr, out, depth + 1)
+            _encode_expanded(f"{prefix}@{idx} ", arr, out, depth + 1, opts)
 
 
 def _all_primitives(arr: list) -> bool:

{gcf_python-2.1.1 → gcf_python-2.2.1}/tests/test_conformance_v2.py

@@ -98,9 +98,9 @@ def test_conformance(rel_path, data):
         ), f"decode mismatch:\n  got: {got}\n  exp: {data['expected']}"
 
     elif op == "roundtrip":
-        # Encode, verify GCF output matches expected, then decode and verify round-trip.
+        # Encode, verify GCF output matches expected (if provided), then decode and verify round-trip.
         got = encode_generic(data["input"])
-        if isinstance(data["expected"], str):
+        if "expected" in data and isinstance(data["expected"], str):
             assert got == data["expected"], f"encode mismatch:\n  got: {got!r}\n  exp: {data['expected']!r}"
         decoded = decode_generic(got)
         assert _structural_equal(

{gcf_python-2.1.1 → gcf_python-2.2.1}/tests/test_generic.py

@@ -1,6 +1,7 @@
 """Tests for GCF generic encoding."""
 
-from gcf import encode_generic
+import pytest
+from gcf import encode_generic, decode_generic, GenericOptions
 
 
 def test_encode_flat_tabular_list():
@@ -60,15 +61,11 @@ def test_encode_mixed_data():
     }
     output = encode_generic(data)
 
-    # Header includes all fields (v2.0: nested fields in union too).
-    assert "## projects [2]{name,status,config}" in output
-    # Rows with nested data get @id prefix and ^ marker.
-    assert "@0 Alpha|active|^" in output
-    assert "@1 Beta|draft|^" in output
-    # Nested config uses .field {} attachment.
-    assert ".config {}" in output
-    assert "env=" in output
-    assert "region=" in output
+    # Header includes flattened path columns (v3.2).
+    assert '## projects [2]{name,status,"config>env","config>region"}' in output
+    # Rows with flattened data are pure tabular.
+    assert "Alpha|active|prod|us-east" in output
+    assert "Beta|draft|staging|eu-west" in output
 
 
 def test_encode_none_value():
@@ -168,3 +165,52 @@ def test_encode_string_with_pipe():
     data = {"val": "a|b"}
     output = encode_generic(data)
     assert 'val="a|b"' in output
+
+
+def test_no_flatten_option():
+    """noFlatten produces attachment syntax instead of path columns."""
+    data = {
+        "orders": [
+            {"id": "ORD-1", "customer": {"name": "Alice", "email": "alice@co.com"}, "total": 99.99},
+            {"id": "ORD-2", "customer": {"name": "Bob", "email": "bob@co.com"}, "total": 49.99},
+        ],
+    }
+    with_flatten = encode_generic(data)
+    assert "customer>" in with_flatten
+
+    no_flatten = encode_generic(data, GenericOptions(no_flatten=True))
+    assert "customer>" not in no_flatten
+    assert ".customer" in no_flatten
+
+    # Both round-trip.
+    assert decode_generic(with_flatten) == data
+    assert decode_generic(no_flatten) == data
+
+
+GT_EDGE_CASES = [
+    ("literal > key", [{">": 1}, {">": 2}]),
+    ("> at start", [{">foo": "a", "id": 1}, {">foo": "b", "id": 2}]),
+    ("> at end", [{"foo>": "a", "id": 1}, {"foo>": "b", "id": 2}]),
+    ("double >>", [{"a>>b": "x"}, {"a>>b": "y"}]),
+    ("multiple > in key", [{"a>b>c": "x"}, {"a>b>c": "y"}]),
+    ("> field with null", [{"a>b": None, "id": 1}, {"a>b": "hello", "id": 2}]),
+    ("> field with object", [{"a>b": {"x": 1}, "id": 1}, {"a>b": {"x": 2}, "id": 2}]),
+    ("> field with array", [{"a>b": [1, 2], "id": 1}, {"a>b": [3], "id": 2}]),
+    ("all fields have >", [{">": 1, "a>b": 2}, {">": 3, "a>b": 4}]),
+    ("mix of > literal and flattened", [
+        {"id": 1, "x>y": "lit", "nested": {"a": "v1", "b": "v2"}},
+        {"id": 2, "x>y": "lit2", "nested": {"a": "v3", "b": "v4"}},
+    ]),
+    ("> field absent in some rows", [{"id": 1, "a>b": "present"}, {"id": 2}]),
+    ("key looks like flattened path", [{"id": 1, "customer>name": "Alice"}, {"id": 2, "customer>name": "Bob"}]),
+]
+
+
+@pytest.mark.parametrize("name,data", GT_EDGE_CASES)
+def test_gt_field_edge_cases(name, data):
+    """Field names containing > round-trip correctly in both flatten modes."""
+    for no_flatten in (False, True):
+        opts = GenericOptions(no_flatten=no_flatten)
+        encoded = encode_generic(data, opts)
+        decoded = decode_generic(encoded)
+        assert decoded == data, f"{name} (no_flatten={no_flatten}): round-trip mismatch\n  gcf: {encoded!r}"

{gcf_python-2.1.1 → gcf_python-2.2.1}/tests/test_roundtrip_v2.py

@@ -7,7 +7,7 @@ import string
 
 import pytest
 
-from gcf import encode_generic, decode_generic
+from gcf import encode_generic, decode_generic, GenericOptions
 
 ITERATIONS = int(os.environ.get("GCF_ITERATIONS", "100000"))
 
@@ -55,7 +55,7 @@ def _gen_number(r):
     ])()
 
 
-SPECIAL = ' |,="\\#@\n\t~^+-.'
+SPECIAL = ' |,="\\#@\n\t~^+-.>'
 BARE = string.ascii_letters + string.digits
 
 
@@ -218,29 +218,31 @@ def test_random_roundtrip():
     r = _rng(42)
     for i in range(ITERATIONS):
         val = _gen_value(r, 0, 4)
-        gcf = encode_generic(val)
-        decoded = decode_generic(gcf)
-        a = _json_norm(val)
-        b = _json_norm(decoded)
-        assert _structural_equal(a, b), (
-            f"iteration {i}: round-trip mismatch\n"
-            f"  input:   {json.dumps(val)}\n"
-            f"  decoded: {json.dumps(decoded)}\n"
-            f"  gcf:     {gcf!r}"
-        )
+        for no_flatten in (False, True):
+            gcf = encode_generic(val, GenericOptions(no_flatten=no_flatten))
+            decoded = decode_generic(gcf)
+            a = _json_norm(val)
+            b = _json_norm(decoded)
+            assert _structural_equal(a, b), (
+                f"iteration {i} no_flatten={no_flatten}: round-trip mismatch\n"
+                f"  input:   {json.dumps(val)}\n"
+                f"  decoded: {json.dumps(decoded)}\n"
+                f"  gcf:     {gcf!r}"
+            )
 
 
 def test_adversarial_roundtrip():
     r = _rng(99)
     for i in range(ITERATIONS):
         val = _gen_adversarial_value(r, 0, 3)
-        gcf = encode_generic(val)
-        decoded = decode_generic(gcf)
-        a = _json_norm(val)
-        b = _json_norm(decoded)
-        assert _structural_equal(a, b), (
-            f"iteration {i}: round-trip mismatch\n"
-            f"  input:   {json.dumps(val)}\n"
-            f"  decoded: {json.dumps(decoded)}\n"
-            f"  gcf:     {gcf!r}"
-        )
+        for no_flatten in (False, True):
+            gcf = encode_generic(val, GenericOptions(no_flatten=no_flatten))
+            decoded = decode_generic(gcf)
+            a = _json_norm(val)
+            b = _json_norm(decoded)
+            assert _structural_equal(a, b), (
+                f"iteration {i} no_flatten={no_flatten}: round-trip mismatch\n"
+                f"  input:   {json.dumps(val)}\n"
+                f"  decoded: {json.dumps(decoded)}\n"
+                f"  gcf:     {gcf!r}"
+            )