gcf-python 2.3.0__tar.gz → 2.4.0__tar.gz
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- {gcf_python-2.3.0 → gcf_python-2.4.0}/CHANGELOG.md +27 -0
- {gcf_python-2.3.0 → gcf_python-2.4.0}/PKG-INFO +7 -3
- {gcf_python-2.3.0 → gcf_python-2.4.0}/README.md +6 -2
- {gcf_python-2.3.0 → gcf_python-2.4.0}/pyproject.toml +1 -1
- {gcf_python-2.3.0 → gcf_python-2.4.0}/src/gcf/__init__.py +5 -1
- {gcf_python-2.3.0 → gcf_python-2.4.0}/src/gcf/decode_generic.py +21 -1
- gcf_python-2.4.0/src/gcf/delta.py +237 -0
- {gcf_python-2.3.0 → gcf_python-2.4.0}/src/gcf/encode.py +36 -14
- gcf_python-2.4.0/src/gcf/packroot.py +53 -0
- {gcf_python-2.3.0 → gcf_python-2.4.0}/src/gcf/session.py +26 -16
- {gcf_python-2.3.0 → gcf_python-2.4.0}/src/gcf/stream.py +22 -13
- gcf_python-2.4.0/tests/test_conformance_v2.py +422 -0
- {gcf_python-2.3.0 → gcf_python-2.4.0}/tests/test_delta.py +5 -5
- {gcf_python-2.3.0 → gcf_python-2.4.0}/tests/test_encode.py +3 -2
- gcf_python-2.4.0/tests/test_generic_delta_fuzz.py +60 -0
- {gcf_python-2.3.0 → gcf_python-2.4.0}/tests/test_roundtrip.py +5 -5
- {gcf_python-2.3.0 → gcf_python-2.4.0}/tests/test_session.py +3 -2
- gcf_python-2.3.0/src/gcf/delta.py +0 -54
- gcf_python-2.3.0/tests/test_conformance_v2.py +0 -202
- {gcf_python-2.3.0 → gcf_python-2.4.0}/.github/FUNDING.yml +0 -0
- {gcf_python-2.3.0 → gcf_python-2.4.0}/.github/workflows/ci.yml +0 -0
- {gcf_python-2.3.0 → gcf_python-2.4.0}/.github/workflows/publish.yml +0 -0
- {gcf_python-2.3.0 → gcf_python-2.4.0}/.gitignore +0 -0
- {gcf_python-2.3.0 → gcf_python-2.4.0}/LICENSE +0 -0
- {gcf_python-2.3.0 → gcf_python-2.4.0}/assets/gcf-python-diagram.png +0 -0
- {gcf_python-2.3.0 → gcf_python-2.4.0}/src/gcf/__main__.py +0 -0
- {gcf_python-2.3.0 → gcf_python-2.4.0}/src/gcf/cli.py +0 -0
- {gcf_python-2.3.0 → gcf_python-2.4.0}/src/gcf/constants.py +0 -0
- {gcf_python-2.3.0 → gcf_python-2.4.0}/src/gcf/decode.py +0 -0
- {gcf_python-2.3.0 → gcf_python-2.4.0}/src/gcf/generic.py +0 -0
- {gcf_python-2.3.0 → gcf_python-2.4.0}/src/gcf/generic_delta.py +0 -0
- {gcf_python-2.3.0 → gcf_python-2.4.0}/src/gcf/scalar.py +0 -0
- {gcf_python-2.3.0 → gcf_python-2.4.0}/src/gcf/stream_generic.py +0 -0
- {gcf_python-2.3.0 → gcf_python-2.4.0}/src/gcf/types.py +0 -0
- {gcf_python-2.3.0 → gcf_python-2.4.0}/tests/__init__.py +0 -0
- {gcf_python-2.3.0 → gcf_python-2.4.0}/tests/test_decode.py +0 -0
- {gcf_python-2.3.0 → gcf_python-2.4.0}/tests/test_generic.py +0 -0
- {gcf_python-2.3.0 → gcf_python-2.4.0}/tests/test_generic_delta.py +0 -0
- {gcf_python-2.3.0 → gcf_python-2.4.0}/tests/test_generic_delta_session.py +0 -0
- {gcf_python-2.3.0 → gcf_python-2.4.0}/tests/test_roundtrip_v2.py +0 -0
- {gcf_python-2.3.0 → gcf_python-2.4.0}/tests/test_stream.py +0 -0
- {gcf_python-2.3.0 → gcf_python-2.4.0}/tests/test_stream_generic.py +0 -0
- {gcf_python-2.3.0 → gcf_python-2.4.0}/uv.lock +0 -0
|
@@ -1,5 +1,31 @@
|
|
|
1
1
|
# Changelog
|
|
2
2
|
|
|
3
|
+
## v2.4.0 (2026-07-12)
|
|
4
|
+
|
|
5
|
+
### Fixes
|
|
6
|
+
|
|
7
|
+
- The conformance runner now hard-fails on any unhandled operation (instead of silently skipping it) and exercises session, delta, roundtrip, and pack-root fixtures end to end; the graph delta wire decode and verify path is now covered, so no operations remain allow-listed.
|
|
8
|
+
- Implemented the graph delta wire decoder and verifier (`decode_delta` / `verify_delta`): parse a `GCF profile=graph delta=true` wire back into removed/added symbols and edge changes, apply them atomically to a base snapshot, recompute `pack_root`, and reject a wrong `new_root` with `root_mismatch` (SPEC 10.4). The `## added` encoder now emits the trailing `distance` field (SPEC 3.4.1, Section 10.1). The shared `graph-delta` fixtures now run end to end: 001 (encode, gains the trailing distance), 002 (verified apply), 003 (`root_mismatch` rejection).
|
|
9
|
+
- **Session encoding correctness fix.** `encode_with_session` assigned per-response local IDs instead of stable session-global IDs, so the cross-call dedup references (`@N # previously transmitted`) pointed at the wrong symbols, and the header emitted zero-valued `budget`/`tokens`/`edges`. Both are fixed to match the reference; graph session output is now byte-identical across all six SDKs. This had gone undetected because the conformance runner skipped the shared graph-session fixtures (now wired).
|
|
10
|
+
- Added the graph-profile PackRoot (`pack_root(symbols, edges)`, gcf-pack-root-v1, SPEC 10.2): the content-addressed sha256 over canonical, independently-sorted symbol/edge records, byte-identical to gcf-go/rust/typescript/swift/kotlin. The conformance runner now exercises the shared `graph-pack-root` fixtures, which it had been skipping (so this primitive was previously unimplemented and untested).
|
|
11
|
+
- Buffered graph encoder now matches the reference byte-for-byte: symbols are ordered by distance then descending score with local IDs assigned in output order, and the header omits `budget`/`tokens`/`edges` when zero (previously symbols kept input order and zero-valued header fields were always emitted). The conformance runner now exercises the shared `graph-encode` fixtures (001-003), which it had been skipping - which is how this divergence went uncaught.
|
|
12
|
+
- Buffered graph encoder: order edges by source ID, then target ID, then edge type (SPEC 16.1), instead of emitting them in input order. Decode-invariant (edges are a set) and does not affect `pack_root` (which sorts edge records independently), so no content addresses change. Pinned by shared fixture `graph-encode/003`. Streaming edges remain in producer-arrival order.
|
|
13
|
+
- Decoder: reject an orphan `.field` attachment (a `.field` whose name is neither a `^`-marked column of its row nor a `>`-containing field name, SPEC 7.4.6.1.4) instead of silently absorbing it as an undeclared extra field. Such a stray attachment previously decoded to a record no encoder produces, silently injecting a field onto the last-parsed row (a lossless round-trip hole); now rejected per SPEC 16.5 (`orphan_attachment`).
|
|
14
|
+
- Decoder: reject an orphan positional inline body (a pipe-delimited line with no eligible `^{}` attachment-marker cell) instead of silently dropping it. The object-body parser previously skipped any unrecognized line, so a stray positional body (e.g. a second `Bob|b@t.com` after a row's one inline cell was filled) vanished with no error (silent data loss); now rejected per SPEC 16.5 (`orphan_inline_attachment`).
|
|
15
|
+
- Graph streaming trailer: the edge count is now always the last `counts` entry, even when the stream has no edges (positional `counts=2,1,0`; labeled `counts=…,edges:0`). A zero-edge stream previously dropped it, violating the SPEC §8.4 / §8.4.1 rule that the edge count is always present and last (the invariant that keeps the positional form unambiguous). The graph trailer is decoder-ignored, so this changes producer output only.
|
|
16
|
+
|
|
17
|
+
### Streaming: opt-in labeled trailer counts (SPEC §8.4.1)
|
|
18
|
+
|
|
19
|
+
- New `labeled_trailer_counts` keyword on `StreamEncoder`. When set, the `##! summary` graph streaming trailer emits `counts=` in the labeled form `label:count` per group (e.g. `counts=targets:2,related:1,edges:3`) instead of the default positional values-only form (`counts=2,1,3`). Default false is byte-identical to prior output.
|
|
20
|
+
- Opt-in and non-breaking: a producer-side comprehension aid for known weak consumers. The trailer counts remain informational (decoder-ignored) in both forms; neither changes the decoded payload. Mirrors the `gcf-go` reference.
|
|
21
|
+
|
|
22
|
+
### Conformance and docs
|
|
23
|
+
|
|
24
|
+
- Streaming graph trailer now emits `distance_N` group counts in pure group-header emission order (dropping a fixed `targets,related,extended` prefix), matching the other SDKs and deterministic per SPEC 16.1. Byte-identical for contract-conformant (ascending-distance) input; pinned by shared fixtures `streaming-v2/010`–`011`.
|
|
25
|
+
- The conformance runner now executes the `graph-stream-encode` fixtures (streaming-encode parity, previously decode-only): fixture 004 (positional trailer) and 005 (labeled trailer).
|
|
26
|
+
- README: corrected the streaming example trailer from the defunct `## _summary … sections=` to the real `##! summary … counts=`; README now leads with the project diagram.
|
|
27
|
+
- Added a generic-delta fuzz test (decoder never crashes; string round-trip).
|
|
28
|
+
|
|
3
29
|
## v2.3.0 (2026-07-12)
|
|
4
30
|
|
|
5
31
|
### Generic-profile delta encoding (SPEC §10a)
|
|
@@ -18,6 +44,7 @@
|
|
|
18
44
|
- Unit suite mirroring `gcf-go`: self-proving round-trip (diff -> encode -> apply -> recomputed root), determinism / row-order invariance, no-type-collision canonicalization, every invariant/error path, full-payload wire round-trip, the complete server -> wire -> consumer end-to-end loop, and malformed-wire-fails-closed.
|
|
19
45
|
- Conformance runner support for `generic-pack-root`, `generic-delta`, `generic-delta-verify`, `generic-delta-decode` (12 shared fixtures); verified to produce identical pack roots and delta wire to `gcf-go`.
|
|
20
46
|
- Session helper suite (`test_generic_delta_session.py`) mirroring `gcf-go`: FixedN cadence pattern, size-guard triggering, schema-change forced full, FixedN(15)-over-30-turns count, and the load-bearing consumer-stays-in-sync check under both policies. Conformance runner support for `generic-delta-session` (3 shared fixtures: fixed-N, size-guard, schema-change).
|
|
47
|
+
- Generic-delta fuzz (`test_generic_delta_fuzz.py`), mirroring `gcf-go`: the decoder never crashes on arbitrary/mutated input, and arbitrary UTF-8 string cells (including multi-byte and control characters) survive the full-wire round-trip with the pack root preserved.
|
|
21
48
|
|
|
22
49
|
## v2.2.2 (2026-07-10)
|
|
23
50
|
|
|
@@ -1,6 +1,6 @@
|
|
|
1
1
|
Metadata-Version: 2.4
|
|
2
2
|
Name: gcf-python
|
|
3
|
-
Version: 2.
|
|
3
|
+
Version: 2.4.0
|
|
4
4
|
Summary: The AI-native wire format for structured data. 50-92% fewer tokens than JSON, with multi-turn delta encoding for agent loops. 100% comprehension on every frontier model. Zero dependencies.
|
|
5
5
|
Project-URL: Homepage, https://github.com/blackwell-systems/gcf-python
|
|
6
6
|
Project-URL: Documentation, https://gcformat.com/
|
|
@@ -23,6 +23,10 @@ Classifier: Typing :: Typed
|
|
|
23
23
|
Requires-Python: >=3.9
|
|
24
24
|
Description-Content-Type: text/markdown
|
|
25
25
|
|
|
26
|
+
<p align="center">
|
|
27
|
+
<img src="assets/gcf-python-diagram.png" alt="gcf-python" width="100%">
|
|
28
|
+
</p>
|
|
29
|
+
|
|
26
30
|
<p align="center">
|
|
27
31
|
<a href="https://github.com/blackwell-systems"><img src="https://raw.githubusercontent.com/blackwell-systems/blackwell-docs-theme/main/badge-trademark.svg" alt="Blackwell Systems"></a>
|
|
28
32
|
<a href="LICENSE"><img src="https://img.shields.io/badge/license-MIT-blue.svg" alt="License"></a>
|
|
@@ -119,7 +123,7 @@ enc = StreamEncoder(sys.stdout, "context_for_task", token_budget=5000)
|
|
|
119
123
|
enc.write_symbol(Symbol(qualified_name="pkg.Auth", kind="function", score=0.95, provenance="lsp", distance=0))
|
|
120
124
|
enc.write_symbol(Symbol(qualified_name="pkg.Server", kind="function", score=0.60, provenance="lsp", distance=1))
|
|
121
125
|
enc.write_edge(Edge(source="pkg.Server", target="pkg.Auth", edge_type="calls"))
|
|
122
|
-
enc.close() # emits
|
|
126
|
+
enc.close() # emits ##! summary trailer
|
|
123
127
|
```
|
|
124
128
|
|
|
125
129
|
Output:
|
|
@@ -131,7 +135,7 @@ GCF tool=context_for_task budget=5000
|
|
|
131
135
|
@1 fn pkg.Server 0.60 lsp
|
|
132
136
|
## edges [?]
|
|
133
137
|
@0<@1 calls
|
|
134
|
-
|
|
138
|
+
##! summary symbols=2 edges=1 counts=1,1,1
|
|
135
139
|
```
|
|
136
140
|
|
|
137
141
|
The writer is any object with a `write(s: str)` method. Thread-safe. Standard `decode()` handles streaming output with no changes.
|
|
@@ -1,3 +1,7 @@
|
|
|
1
|
+
<p align="center">
|
|
2
|
+
<img src="assets/gcf-python-diagram.png" alt="gcf-python" width="100%">
|
|
3
|
+
</p>
|
|
4
|
+
|
|
1
5
|
<p align="center">
|
|
2
6
|
<a href="https://github.com/blackwell-systems"><img src="https://raw.githubusercontent.com/blackwell-systems/blackwell-docs-theme/main/badge-trademark.svg" alt="Blackwell Systems"></a>
|
|
3
7
|
<a href="LICENSE"><img src="https://img.shields.io/badge/license-MIT-blue.svg" alt="License"></a>
|
|
@@ -94,7 +98,7 @@ enc = StreamEncoder(sys.stdout, "context_for_task", token_budget=5000)
|
|
|
94
98
|
enc.write_symbol(Symbol(qualified_name="pkg.Auth", kind="function", score=0.95, provenance="lsp", distance=0))
|
|
95
99
|
enc.write_symbol(Symbol(qualified_name="pkg.Server", kind="function", score=0.60, provenance="lsp", distance=1))
|
|
96
100
|
enc.write_edge(Edge(source="pkg.Server", target="pkg.Auth", edge_type="calls"))
|
|
97
|
-
enc.close() # emits
|
|
101
|
+
enc.close() # emits ##! summary trailer
|
|
98
102
|
```
|
|
99
103
|
|
|
100
104
|
Output:
|
|
@@ -106,7 +110,7 @@ GCF tool=context_for_task budget=5000
|
|
|
106
110
|
@1 fn pkg.Server 0.60 lsp
|
|
107
111
|
## edges [?]
|
|
108
112
|
@0<@1 calls
|
|
109
|
-
|
|
113
|
+
##! summary symbols=2 edges=1 counts=1,1,1
|
|
110
114
|
```
|
|
111
115
|
|
|
112
116
|
The writer is any object with a `write(s: str)` method. Thread-safe. Standard `decode()` handles streaming output with no changes.
|
|
@@ -4,7 +4,7 @@ build-backend = "hatchling.build"
|
|
|
4
4
|
|
|
5
5
|
[project]
|
|
6
6
|
name = "gcf-python"
|
|
7
|
-
version = "2.
|
|
7
|
+
version = "2.4.0"
|
|
8
8
|
description = "The AI-native wire format for structured data. 50-92% fewer tokens than JSON, with multi-turn delta encoding for agent loops. 100% comprehension on every frontier model. Zero dependencies."
|
|
9
9
|
readme = "README.md"
|
|
10
10
|
license = {text = "MIT"}
|
|
@@ -36,7 +36,7 @@ Specification: https://github.com/blackwell-systems/gcf
|
|
|
36
36
|
|
|
37
37
|
from .constants import KIND_ABBREV, KIND_EXPAND
|
|
38
38
|
from .decode import DecodeError, decode
|
|
39
|
-
from .delta import encode_delta
|
|
39
|
+
from .delta import decode_delta, encode_delta, verify_delta
|
|
40
40
|
from .encode import encode
|
|
41
41
|
from .generic import encode_generic, GenericOptions
|
|
42
42
|
from .generic_delta import (
|
|
@@ -56,6 +56,7 @@ from .generic_delta import (
|
|
|
56
56
|
size_guard,
|
|
57
57
|
DEFAULT_REANCHOR_N,
|
|
58
58
|
)
|
|
59
|
+
from .packroot import pack_root
|
|
59
60
|
from .session import Session, encode_with_session
|
|
60
61
|
from .decode_generic import decode_generic
|
|
61
62
|
from .stream import StreamEncoder
|
|
@@ -75,15 +76,18 @@ __all__ = [
|
|
|
75
76
|
"StreamEncoder",
|
|
76
77
|
"Symbol",
|
|
77
78
|
"decode",
|
|
79
|
+
"decode_delta",
|
|
78
80
|
"decode_generic",
|
|
79
81
|
"encode",
|
|
80
82
|
"encode_delta",
|
|
83
|
+
"verify_delta",
|
|
81
84
|
"encode_generic",
|
|
82
85
|
"GenericOptions",
|
|
83
86
|
"encode_with_session",
|
|
84
87
|
"GenericSet",
|
|
85
88
|
"GenericDeltaPayload",
|
|
86
89
|
"generic_pack_root",
|
|
90
|
+
"pack_root",
|
|
87
91
|
"diff_generic_sets",
|
|
88
92
|
"encode_generic_full",
|
|
89
93
|
"encode_generic_delta",
|
|
@@ -177,7 +177,14 @@ def _parse_object_body(
|
|
|
177
177
|
i += 1
|
|
178
178
|
continue
|
|
179
179
|
|
|
180
|
-
|
|
180
|
+
# An object-body line that is not a `## ` section, a `key=value` field, or
|
|
181
|
+
# an inline array is not valid content and MUST NOT be silently skipped
|
|
182
|
+
# (that dropped data, a lossless round-trip hole). A pipe-delimited line is
|
|
183
|
+
# a stray positional inline body with no eligible `^` cell (SPEC 16.5,
|
|
184
|
+
# orphan_inline_attachment); any other unrecognized line is likewise rejected.
|
|
185
|
+
if "|" in content:
|
|
186
|
+
raise ValueError(f"orphan_inline_attachment: {content}")
|
|
187
|
+
raise ValueError(f"invalid_line: unexpected content in object body: {content!r}")
|
|
181
188
|
return i - start
|
|
182
189
|
|
|
183
190
|
|
|
@@ -521,6 +528,11 @@ def _parse_tabular_body(
|
|
|
521
528
|
if row_has_id:
|
|
522
529
|
inline_idx = 0
|
|
523
530
|
|
|
531
|
+
# Columns that carry a `^` marker cell in this row legitimately expect
|
|
532
|
+
# a `.field` body. Any other `.field` is an orphan (Section 16.5) unless
|
|
533
|
+
# its name contains `>` (the flatten-fallback attachment, Section 7.4.6.1.4).
|
|
534
|
+
expected_att = set(traditional_att_fields) | set(inline_att_fields)
|
|
535
|
+
|
|
524
536
|
while i < len(lines):
|
|
525
537
|
a_line = lines[i]
|
|
526
538
|
a_content: str | None = None
|
|
@@ -541,6 +553,14 @@ def _parse_tabular_body(
|
|
|
541
553
|
att_name, after_name = _parse_attachment_name(rest)
|
|
542
554
|
after_name_stripped = after_name.lstrip()
|
|
543
555
|
|
|
556
|
+
# Orphan attachment: a `.field` with no matching `^` cell in this
|
|
557
|
+
# row is only legitimate for a `>`-named field (Section 7.4.6.1.4).
|
|
558
|
+
# Any other unmatched attachment is rejected rather than silently
|
|
559
|
+
# injected as an undeclared extra field, which would decode to a
|
|
560
|
+
# record no encoder produces (Section 16.5, lossless round-trip).
|
|
561
|
+
if att_name not in expected_att and ">" not in att_name:
|
|
562
|
+
raise ValueError(f"orphan_attachment: {att_name}")
|
|
563
|
+
|
|
544
564
|
# Prefixed inline data.
|
|
545
565
|
ifs = inline_schemas.get(att_name)
|
|
546
566
|
if ifs and not after_name_stripped.startswith("{}") and not after_name_stripped.startswith("["):
|
|
@@ -0,0 +1,237 @@
|
|
|
1
|
+
"""GCF delta encoding: only added/removed symbols for incremental delivery."""
|
|
2
|
+
|
|
3
|
+
from .constants import KIND_ABBREV, KIND_EXPAND
|
|
4
|
+
from .packroot import pack_root
|
|
5
|
+
from .types import DeltaPayload, Edge, Symbol
|
|
6
|
+
|
|
7
|
+
|
|
8
|
+
def encode_delta(d: DeltaPayload) -> str:
|
|
9
|
+
"""Encode a DeltaPayload into GCF delta format.
|
|
10
|
+
|
|
11
|
+
Args:
|
|
12
|
+
d: The delta payload to encode.
|
|
13
|
+
|
|
14
|
+
Returns:
|
|
15
|
+
GCF delta-formatted text string.
|
|
16
|
+
"""
|
|
17
|
+
parts: list[str] = []
|
|
18
|
+
|
|
19
|
+
# Header.
|
|
20
|
+
savings = 0.0
|
|
21
|
+
if d.full_tokens > 0:
|
|
22
|
+
savings = 100.0 * (1.0 - d.delta_tokens / d.full_tokens)
|
|
23
|
+
|
|
24
|
+
parts.append(
|
|
25
|
+
f"GCF profile=graph tool={d.tool} delta=true base_root={d.base_root} "
|
|
26
|
+
f"new_root={d.new_root} tokens={d.delta_tokens} savings={savings:.0f}%"
|
|
27
|
+
)
|
|
28
|
+
|
|
29
|
+
# Removed symbols: short references (consumer already has the full declaration).
|
|
30
|
+
if d.removed:
|
|
31
|
+
parts.append("## removed")
|
|
32
|
+
for s in d.removed:
|
|
33
|
+
kind = KIND_ABBREV.get(s.kind, s.kind)
|
|
34
|
+
parts.append(f"{kind} {s.qualified_name}")
|
|
35
|
+
|
|
36
|
+
# Added symbols: full declarations (consumer doesn't have these).
|
|
37
|
+
if d.added:
|
|
38
|
+
parts.append("## added")
|
|
39
|
+
for i, s in enumerate(d.added):
|
|
40
|
+
kind = KIND_ABBREV.get(s.kind, s.kind)
|
|
41
|
+
parts.append(
|
|
42
|
+
f"@{i} {kind} {s.qualified_name} {s.score:.2f} {s.provenance} {s.distance}"
|
|
43
|
+
)
|
|
44
|
+
|
|
45
|
+
# Removed edges.
|
|
46
|
+
if d.removed_edges:
|
|
47
|
+
parts.append("## edges_removed")
|
|
48
|
+
for e in d.removed_edges:
|
|
49
|
+
parts.append(f"{e.source} -> {e.target} {e.edge_type}")
|
|
50
|
+
|
|
51
|
+
# Added edges.
|
|
52
|
+
if d.added_edges:
|
|
53
|
+
parts.append("## edges_added")
|
|
54
|
+
for e in d.added_edges:
|
|
55
|
+
parts.append(f"{e.source} -> {e.target} {e.edge_type}")
|
|
56
|
+
|
|
57
|
+
return "\n".join(parts) + "\n"
|
|
58
|
+
|
|
59
|
+
|
|
60
|
+
def _expand_kind(k: str) -> str:
|
|
61
|
+
"""Reverse a kind abbreviation to its full form (identity if unknown)."""
|
|
62
|
+
return KIND_EXPAND.get(k, k)
|
|
63
|
+
|
|
64
|
+
|
|
65
|
+
def _parse_delta_edge(line: str) -> Edge:
|
|
66
|
+
"""Parse a `source -> target type` delta edge line."""
|
|
67
|
+
idx = line.find(" -> ")
|
|
68
|
+
if idx < 0:
|
|
69
|
+
raise ValueError(f"malformed_delta: edge line missing ' -> ': {line!r}")
|
|
70
|
+
source = line[:idx]
|
|
71
|
+
rest = line[idx + 4 :].split()
|
|
72
|
+
if len(rest) != 2:
|
|
73
|
+
raise ValueError(
|
|
74
|
+
f"malformed_delta: edge line {line!r} must be 'source -> target type'"
|
|
75
|
+
)
|
|
76
|
+
return Edge(source=source, target=rest[0], edge_type=rest[1])
|
|
77
|
+
|
|
78
|
+
|
|
79
|
+
def decode_delta(wire: str) -> DeltaPayload:
|
|
80
|
+
"""Parse a GCF graph delta wire payload back into a DeltaPayload.
|
|
81
|
+
|
|
82
|
+
Kind abbreviations on removed/added lines are expanded to their full form so the
|
|
83
|
+
result matches a base snapshot's symbol identities. Raises ValueError containing
|
|
84
|
+
``malformed_delta`` on bad lines or unknown sections.
|
|
85
|
+
"""
|
|
86
|
+
lines = wire.rstrip("\n").split("\n")
|
|
87
|
+
if not lines or lines[0] == "":
|
|
88
|
+
raise ValueError("missing_header: empty delta payload")
|
|
89
|
+
header = lines[0].rstrip("\r")
|
|
90
|
+
if not header.startswith("GCF profile=graph"):
|
|
91
|
+
raise ValueError(
|
|
92
|
+
"missing_profile: delta header must begin with 'GCF profile=graph'"
|
|
93
|
+
)
|
|
94
|
+
|
|
95
|
+
d = DeltaPayload()
|
|
96
|
+
for field in header.split():
|
|
97
|
+
kv = field.split("=", 1)
|
|
98
|
+
if len(kv) != 2:
|
|
99
|
+
continue
|
|
100
|
+
key, value = kv
|
|
101
|
+
if key == "tool":
|
|
102
|
+
d.tool = value
|
|
103
|
+
elif key == "base_root":
|
|
104
|
+
d.base_root = value
|
|
105
|
+
elif key == "new_root":
|
|
106
|
+
d.new_root = value
|
|
107
|
+
|
|
108
|
+
section = ""
|
|
109
|
+
for raw in lines[1:]:
|
|
110
|
+
line = raw.rstrip("\r")
|
|
111
|
+
if line == "":
|
|
112
|
+
continue
|
|
113
|
+
if line.startswith("## "):
|
|
114
|
+
section = line[3:].strip()
|
|
115
|
+
if section not in ("removed", "added", "edges_removed", "edges_added"):
|
|
116
|
+
raise ValueError(f"malformed_delta: unknown section {section!r}")
|
|
117
|
+
continue
|
|
118
|
+
if section == "removed":
|
|
119
|
+
parts = line.split()
|
|
120
|
+
if len(parts) != 2:
|
|
121
|
+
raise ValueError(
|
|
122
|
+
f"malformed_delta: removed line {line!r} must be 'kind qname'"
|
|
123
|
+
)
|
|
124
|
+
d.removed.append(
|
|
125
|
+
Symbol(kind=_expand_kind(parts[0]), qualified_name=parts[1])
|
|
126
|
+
)
|
|
127
|
+
elif section == "added":
|
|
128
|
+
parts = line.split()
|
|
129
|
+
if len(parts) != 6:
|
|
130
|
+
raise ValueError(
|
|
131
|
+
f"malformed_delta: added line {line!r} must be "
|
|
132
|
+
"'@id kind qname score provenance distance'"
|
|
133
|
+
)
|
|
134
|
+
try:
|
|
135
|
+
score = float(parts[3])
|
|
136
|
+
except ValueError:
|
|
137
|
+
raise ValueError(f"malformed_delta: invalid added score {parts[3]!r}")
|
|
138
|
+
try:
|
|
139
|
+
dist = int(parts[5])
|
|
140
|
+
except ValueError:
|
|
141
|
+
raise ValueError(
|
|
142
|
+
f"malformed_delta: invalid added distance {parts[5]!r}"
|
|
143
|
+
)
|
|
144
|
+
d.added.append(
|
|
145
|
+
Symbol(
|
|
146
|
+
kind=_expand_kind(parts[1]),
|
|
147
|
+
qualified_name=parts[2],
|
|
148
|
+
score=score,
|
|
149
|
+
provenance=parts[4],
|
|
150
|
+
distance=dist,
|
|
151
|
+
)
|
|
152
|
+
)
|
|
153
|
+
elif section in ("edges_removed", "edges_added"):
|
|
154
|
+
e = _parse_delta_edge(line)
|
|
155
|
+
if section == "edges_removed":
|
|
156
|
+
d.removed_edges.append(e)
|
|
157
|
+
else:
|
|
158
|
+
d.added_edges.append(e)
|
|
159
|
+
else:
|
|
160
|
+
raise ValueError(
|
|
161
|
+
f"malformed_delta: data line {line!r} before any section header"
|
|
162
|
+
)
|
|
163
|
+
return d
|
|
164
|
+
|
|
165
|
+
|
|
166
|
+
def verify_delta(
|
|
167
|
+
base_symbols: list[Symbol],
|
|
168
|
+
base_edges: list[Edge],
|
|
169
|
+
removed: list[Symbol],
|
|
170
|
+
added: list[Symbol],
|
|
171
|
+
removed_edges: list[Edge],
|
|
172
|
+
added_edges: list[Edge],
|
|
173
|
+
expected_new_root: str,
|
|
174
|
+
) -> tuple[list[Symbol], list[Edge]]:
|
|
175
|
+
"""Apply a delta to a base snapshot and verify the resulting pack root.
|
|
176
|
+
|
|
177
|
+
Symbols are matched by identity ``(kind, qualified_name)``; edges by
|
|
178
|
+
``(source, target, edge_type)``. Raises ValueError containing ``delta_invalid``
|
|
179
|
+
when removing a symbol/edge that does not exist or adding one that already exists,
|
|
180
|
+
and ``root_mismatch`` when the recomputed pack root differs from
|
|
181
|
+
``expected_new_root``. On success returns the applied ``(symbols, edges)``.
|
|
182
|
+
"""
|
|
183
|
+
sym_map: dict[tuple[str, str], Symbol] = {}
|
|
184
|
+
for s in base_symbols:
|
|
185
|
+
sym_map[(s.kind, s.qualified_name)] = s
|
|
186
|
+
|
|
187
|
+
for s in removed:
|
|
188
|
+
key = (s.kind, s.qualified_name)
|
|
189
|
+
if key not in sym_map:
|
|
190
|
+
raise ValueError(
|
|
191
|
+
f"delta_invalid: removing symbol {s.kind} {s.qualified_name} "
|
|
192
|
+
"that does not exist in base"
|
|
193
|
+
)
|
|
194
|
+
del sym_map[key]
|
|
195
|
+
|
|
196
|
+
for s in added:
|
|
197
|
+
key = (s.kind, s.qualified_name)
|
|
198
|
+
if key in sym_map:
|
|
199
|
+
raise ValueError(
|
|
200
|
+
f"delta_invalid: adding symbol {s.kind} {s.qualified_name} "
|
|
201
|
+
"that already exists"
|
|
202
|
+
)
|
|
203
|
+
sym_map[key] = s
|
|
204
|
+
|
|
205
|
+
result_symbols = list(sym_map.values())
|
|
206
|
+
|
|
207
|
+
edge_map: dict[tuple[str, str, str], Edge] = {}
|
|
208
|
+
for e in base_edges:
|
|
209
|
+
edge_map[(e.source, e.target, e.edge_type)] = e
|
|
210
|
+
|
|
211
|
+
for e in removed_edges:
|
|
212
|
+
key = (e.source, e.target, e.edge_type)
|
|
213
|
+
if key not in edge_map:
|
|
214
|
+
raise ValueError(
|
|
215
|
+
f"delta_invalid: removing edge {e.source} -> {e.target} "
|
|
216
|
+
f"{e.edge_type} that does not exist"
|
|
217
|
+
)
|
|
218
|
+
del edge_map[key]
|
|
219
|
+
|
|
220
|
+
for e in added_edges:
|
|
221
|
+
key = (e.source, e.target, e.edge_type)
|
|
222
|
+
if key in edge_map:
|
|
223
|
+
raise ValueError(
|
|
224
|
+
f"delta_invalid: adding edge {e.source} -> {e.target} "
|
|
225
|
+
f"{e.edge_type} that already exists"
|
|
226
|
+
)
|
|
227
|
+
edge_map[key] = e
|
|
228
|
+
|
|
229
|
+
result_edges = list(edge_map.values())
|
|
230
|
+
|
|
231
|
+
computed_root = pack_root(result_symbols, result_edges)
|
|
232
|
+
if computed_root != expected_new_root:
|
|
233
|
+
raise ValueError(
|
|
234
|
+
f"root_mismatch: computed {computed_root}, expected {expected_new_root}"
|
|
235
|
+
)
|
|
236
|
+
|
|
237
|
+
return result_symbols, result_edges
|
|
@@ -17,10 +17,16 @@ def encode(p: Payload) -> str:
|
|
|
17
17
|
"""
|
|
18
18
|
parts: list[str] = []
|
|
19
19
|
|
|
20
|
-
#
|
|
20
|
+
# Group symbols by distance (sorted by score descending within each group),
|
|
21
|
+
# then assign local IDs in output order so they are sequential in the wire
|
|
22
|
+
# (SPEC 16.1).
|
|
23
|
+
groups = _group_by_distance(p.symbols)
|
|
21
24
|
sym_index: dict[str, int] = {}
|
|
22
|
-
|
|
23
|
-
|
|
25
|
+
next_id = 0
|
|
26
|
+
for _distance, g_symbols in groups:
|
|
27
|
+
for s in g_symbols:
|
|
28
|
+
sym_index[s.qualified_name] = next_id
|
|
29
|
+
next_id += 1
|
|
24
30
|
|
|
25
31
|
# Count valid edges (both endpoints in symbol index).
|
|
26
32
|
valid_edges = sum(
|
|
@@ -28,14 +34,20 @@ def encode(p: Payload) -> str:
|
|
|
28
34
|
if e.source in sym_index and e.target in sym_index
|
|
29
35
|
)
|
|
30
36
|
|
|
31
|
-
# Header line.
|
|
32
|
-
|
|
37
|
+
# Header line (SPEC 16.1): omit budget/tokens/edges when zero, matching the
|
|
38
|
+
# reference encoder.
|
|
39
|
+
header = f"GCF profile=graph tool={p.tool}"
|
|
40
|
+
if p.token_budget > 0:
|
|
41
|
+
header += f" budget={p.token_budget}"
|
|
42
|
+
if p.tokens_used > 0:
|
|
43
|
+
header += f" tokens={p.tokens_used}"
|
|
44
|
+
header += f" symbols={len(p.symbols)}"
|
|
45
|
+
if valid_edges > 0:
|
|
46
|
+
header += f" edges={valid_edges}"
|
|
33
47
|
if p.pack_root:
|
|
34
48
|
header += f" pack_root={p.pack_root}"
|
|
35
49
|
parts.append(header)
|
|
36
50
|
|
|
37
|
-
# Group symbols by distance.
|
|
38
|
-
groups = _group_by_distance(p.symbols)
|
|
39
51
|
group_names = ["targets", "related", "extended"]
|
|
40
52
|
|
|
41
53
|
for g_distance, g_symbols in groups:
|
|
@@ -52,17 +64,25 @@ def encode(p: Payload) -> str:
|
|
|
52
64
|
kind = KIND_ABBREV.get(s.kind, s.kind)
|
|
53
65
|
parts.append(f"@{idx} {kind} {s.qualified_name} {s.score:.2f} {s.provenance}")
|
|
54
66
|
|
|
55
|
-
# Edges section.
|
|
67
|
+
# Edges section. Order edges by source ID then target ID (then edge type
|
|
68
|
+
# for parallel edges) so the wire is canonical regardless of the order
|
|
69
|
+
# edges were provided (SPEC 16.1). Edge reordering is decode-invariant
|
|
70
|
+
# (edges are a set) and does not affect pack_root, which sorts edge records
|
|
71
|
+
# independently.
|
|
56
72
|
if p.edges:
|
|
57
|
-
|
|
73
|
+
resolved: list[tuple[int, int, str, str]] = []
|
|
58
74
|
for e in p.edges:
|
|
59
75
|
src_idx = sym_index.get(e.source)
|
|
60
76
|
tgt_idx = sym_index.get(e.target)
|
|
61
77
|
if src_idx is None or tgt_idx is None:
|
|
62
78
|
continue
|
|
63
|
-
|
|
64
|
-
|
|
65
|
-
|
|
79
|
+
resolved.append((src_idx, tgt_idx, e.edge_type, e.status))
|
|
80
|
+
resolved.sort(key=lambda r: (r[0], r[1], r[2]))
|
|
81
|
+
edge_lines: list[str] = []
|
|
82
|
+
for src_idx, tgt_idx, edge_type, status in resolved:
|
|
83
|
+
line = f"@{tgt_idx}<@{src_idx} {edge_type}"
|
|
84
|
+
if status and status != "unchanged":
|
|
85
|
+
line += f" {status}"
|
|
66
86
|
edge_lines.append(line)
|
|
67
87
|
parts.append(f"## edges [{len(edge_lines)}]")
|
|
68
88
|
parts.extend(edge_lines)
|
|
@@ -71,15 +91,17 @@ def encode(p: Payload) -> str:
|
|
|
71
91
|
|
|
72
92
|
|
|
73
93
|
def _group_by_distance(symbols: list[Symbol]) -> list[tuple[int, list[Symbol]]]:
|
|
74
|
-
"""Group symbols by distance,
|
|
94
|
+
"""Group symbols by distance ascending, sorted by score descending within each
|
|
95
|
+
group (stable), matching the reference encoder so IDs are assigned canonically."""
|
|
75
96
|
if not symbols:
|
|
76
97
|
return []
|
|
77
98
|
|
|
99
|
+
ordered = sorted(symbols, key=lambda s: (s.distance, -s.score))
|
|
78
100
|
groups: list[tuple[int, list[Symbol]]] = []
|
|
79
101
|
current_distance: int | None = None
|
|
80
102
|
current_symbols: list[Symbol] = []
|
|
81
103
|
|
|
82
|
-
for s in
|
|
104
|
+
for s in ordered:
|
|
83
105
|
if current_distance is None or current_distance != s.distance:
|
|
84
106
|
if current_symbols:
|
|
85
107
|
groups.append((current_distance, current_symbols)) # type: ignore[arg-type]
|
|
@@ -0,0 +1,53 @@
|
|
|
1
|
+
"""GCF graph-profile pack root (gcf-pack-root-v1, SPEC Section 10.2).
|
|
2
|
+
|
|
3
|
+
Computes the canonical pack root hash for a graph snapshot (symbols + edges).
|
|
4
|
+
Mirrors the gcf-go reference implementation (packroot.go); the shared conformance
|
|
5
|
+
fixtures (graph-pack-root/) hold both to identical bytes and hashes.
|
|
6
|
+
"""
|
|
7
|
+
|
|
8
|
+
from __future__ import annotations
|
|
9
|
+
|
|
10
|
+
import hashlib
|
|
11
|
+
|
|
12
|
+
from .constants import KIND_ABBREV
|
|
13
|
+
from .scalar import format_number
|
|
14
|
+
from .types import Edge, Symbol
|
|
15
|
+
|
|
16
|
+
|
|
17
|
+
def pack_root(symbols: list[Symbol], edges: list[Edge]) -> str:
|
|
18
|
+
"""Canonical pack root for a graph snapshot (gcf-pack-root-v1, graph profile, 10.2).
|
|
19
|
+
|
|
20
|
+
Symbol and edge records are sorted independently by unsigned UTF-8 byte order,
|
|
21
|
+
then concatenated (all symbols, then all edges) and hashed with SHA-256. Two
|
|
22
|
+
implementations given the same logical graph MUST produce the same result.
|
|
23
|
+
"""
|
|
24
|
+
# Build canonical symbol records.
|
|
25
|
+
sym_records: list[str] = []
|
|
26
|
+
for s in symbols:
|
|
27
|
+
kind = KIND_ABBREV.get(s.kind, s.kind)
|
|
28
|
+
score = format_number(float(s.score))
|
|
29
|
+
sym_records.append(
|
|
30
|
+
f"S\t{kind}\t{s.qualified_name}\t{score}\t{s.provenance}\t{s.distance}\n"
|
|
31
|
+
)
|
|
32
|
+
|
|
33
|
+
# Map qualified_name -> kind abbreviation for edge endpoint resolution.
|
|
34
|
+
sym_kind_map: dict[str, str] = {}
|
|
35
|
+
for s in symbols:
|
|
36
|
+
sym_kind_map[s.qualified_name] = KIND_ABBREV.get(s.kind, s.kind)
|
|
37
|
+
|
|
38
|
+
# Build canonical edge records.
|
|
39
|
+
edge_records: list[str] = []
|
|
40
|
+
for e in edges:
|
|
41
|
+
src_kind = sym_kind_map.get(e.source, "")
|
|
42
|
+
tgt_kind = sym_kind_map.get(e.target, "")
|
|
43
|
+
edge_records.append(
|
|
44
|
+
f"E\t{src_kind}\t{e.source}\t{tgt_kind}\t{e.target}\t{e.edge_type}\n"
|
|
45
|
+
)
|
|
46
|
+
|
|
47
|
+
# Sort independently by unsigned UTF-8 byte order.
|
|
48
|
+
sym_records.sort(key=lambda r: r.encode("utf-8"))
|
|
49
|
+
edge_records.sort(key=lambda r: r.encode("utf-8"))
|
|
50
|
+
|
|
51
|
+
canonical = "".join(sym_records) + "".join(edge_records)
|
|
52
|
+
digest = hashlib.sha256(canonical.encode("utf-8")).hexdigest()
|
|
53
|
+
return "sha256:" + digest
|
|
@@ -77,22 +77,32 @@ def encode_with_session(p: Payload, sess: Session | None = None) -> str:
|
|
|
77
77
|
|
|
78
78
|
parts: list[str] = []
|
|
79
79
|
|
|
80
|
-
# Build
|
|
80
|
+
# Build session-stable ID mapping (matching the reference encoder):
|
|
81
|
+
# previously transmitted symbols keep their existing session-global ID;
|
|
82
|
+
# new symbols get the next available session IDs in payload order. These
|
|
83
|
+
# are NOT per-call local indices, so IDs stay stable across calls within a
|
|
84
|
+
# session (see graph-session conformance fixtures).
|
|
81
85
|
local_index: dict[str, int] = {}
|
|
82
|
-
for
|
|
83
|
-
|
|
84
|
-
|
|
85
|
-
|
|
86
|
-
|
|
87
|
-
|
|
88
|
-
|
|
89
|
-
|
|
90
|
-
|
|
91
|
-
# Header with session=true marker.
|
|
92
|
-
|
|
93
|
-
|
|
94
|
-
|
|
95
|
-
|
|
86
|
+
for s in p.symbols:
|
|
87
|
+
if sess.transmitted(s.qualified_name):
|
|
88
|
+
local_index[s.qualified_name] = sess.get_id(s.qualified_name)
|
|
89
|
+
next_new = sess.size()
|
|
90
|
+
for s in p.symbols:
|
|
91
|
+
if not sess.transmitted(s.qualified_name):
|
|
92
|
+
local_index[s.qualified_name] = next_new
|
|
93
|
+
next_new += 1
|
|
94
|
+
|
|
95
|
+
# Header with session=true marker. Omit budget/tokens/edges when zero,
|
|
96
|
+
# matching the reference encoder.
|
|
97
|
+
header = f"GCF profile=graph tool={p.tool}"
|
|
98
|
+
if p.token_budget > 0:
|
|
99
|
+
header += f" budget={p.token_budget}"
|
|
100
|
+
if p.tokens_used > 0:
|
|
101
|
+
header += f" tokens={p.tokens_used}"
|
|
102
|
+
header += f" symbols={len(p.symbols)}"
|
|
103
|
+
if p.edges:
|
|
104
|
+
header += f" edges={len(p.edges)}"
|
|
105
|
+
header += " session=true"
|
|
96
106
|
if p.pack_root:
|
|
97
107
|
header += f" pack_root={p.pack_root}"
|
|
98
108
|
parts.append(header)
|
|
@@ -128,7 +138,7 @@ def encode_with_session(p: Payload, sess: Session | None = None) -> str:
|
|
|
128
138
|
|
|
129
139
|
# Edges section.
|
|
130
140
|
if p.edges:
|
|
131
|
-
parts.append(f"## edges [{
|
|
141
|
+
parts.append(f"## edges [{len(p.edges)}]")
|
|
132
142
|
for e in p.edges:
|
|
133
143
|
src_idx = local_index.get(e.source)
|
|
134
144
|
tgt_idx = local_index.get(e.target)
|