idemcore-verify 0.3.0__tar.gz → 0.3.2__tar.gz

This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.

Potentially problematic release.


This version of idemcore-verify might be problematic. Click here for more details.

@@ -1,11 +1,11 @@
1
1
  Metadata-Version: 2.4
2
2
  Name: idemcore-verify
3
- Version: 0.3.0
3
+ Version: 0.3.2
4
4
  Summary: Check Idemcore per-token hash logs: verify, without trusting us, that two machines produced bit-identical LLM inference.
5
5
  Author-email: Don Wick <don.wick@idemcore.net>
6
6
  License-Expression: Apache-2.0
7
7
  Project-URL: Homepage, https://idemcore.net
8
- Project-URL: Paper, https://doi.org/10.5281/zenodo.23851129
8
+ Project-URL: Paper, https://doi.org/10.5281/zenodo.23851128
9
9
  Project-URL: Source, https://github.com/idemcore/idemcore-verifier
10
10
  Project-URL: Issues, https://github.com/idemcore/idemcore-verifier/issues
11
11
  Keywords: reproducibility,deterministic inference,llm,verification,integer inference
@@ -39,8 +39,8 @@ Project site: <https://idemcore.net>
39
39
  Idemcore is an integer-only inference engine: the same model, prompt, and
40
40
  settings give the *same bits* on an x86 laptop, an ARM phone, and NVIDIA GPUs
41
41
  from Turing to Hopper. Every token it scores or generates emits a 64-bit hash
42
- of the full logit vector and a 64-bit hash of the integer sampling table
43
- (CDF). A run log is a list of these hashes plus one chained digest (`ALL`).
42
+ of the full logit vector and a 64-bit hash of the integer token-selection
43
+ table. A run log is a list of these hashes plus one chained digest (`ALL`).
44
44
 
45
45
  This repository is the **open half** of Idemcore:
46
46
 
@@ -109,7 +109,8 @@ receipt to every answer, and `GET /v1/receipts/{id}/log` returns that answer's
109
109
  per-token log. The API serves the public form of the log (`log_format`
110
110
  `idemcore-log/r1-public`): the `pi j h hc` fields of [SPEC.md](SPEC.md) plus the
111
111
  `ALL` line, without `freq` and `total`. The verifier accepts both forms (version
112
- 0.3.0 and later). Access needs an API key: sign in to the console at
112
+ 0.3.0 and later; hash-only batched logs from 0.3.2). The sample logs in
113
+ `samples/` are published in the same hash-only form. Access needs an API key: sign in to the console at
113
114
  <https://console.idemcore.net/#/login> and create one. With a key, save the log and run:
114
115
 
115
116
  ```sh
@@ -141,8 +142,9 @@ Requires Python ≥ 3.9, standard library only.
141
142
 
142
143
  ## What the verifier checks
143
144
 
144
- 1. **Well-formedness** – every line has six fields, hashes are 64-bit hex,
145
- `0 < freq ≤ total < 2^31`, no duplicate positions.
145
+ 1. **Well-formedness** – every line has four fields (hash-only) or six,
146
+ hashes are 64-bit hex, `0 < freq ≤ total < 2^31` when present, no
147
+ duplicate positions.
146
148
  2. **Chain integrity** – it recomputes the `ALL` digest from the per-token
147
149
  lines; editing a single line breaks it.
148
150
  3. **Agreement** – it compares a candidate log against a reference log
@@ -152,7 +154,7 @@ Requires Python ≥ 3.9, standard library only.
152
154
 
153
155
  A match on `h` means the two machines computed the **same integer logits for
154
156
  the whole vocabulary** at that position, not just the same argmax token. A
155
- match on `hc` means they built the same integer sampling distribution, so any
157
+ match on `hc` means they built the same integer token-selection table, so any
156
158
  seeded sampler driven by it makes the same choices.
157
159
 
158
160
  ## Published pairs
@@ -197,7 +199,8 @@ The raw batched logs are in `samples/batched/` and are re-checked by step 3 abov
197
199
 
198
200
  Don Wick, *Idemcore: Bit-Identical LLM Inference from Phones to Datacenter
199
201
  GPUs, with an Open Per-Token Verification Protocol*, preprint, Zenodo,
200
- 2026. DOI: [10.5281/zenodo.23851129](https://doi.org/10.5281/zenodo.23851129).
202
+ 2026, version 1.1. DOI: [10.5281/zenodo.23949090](https://doi.org/10.5281/zenodo.23949090)
203
+ (all versions: [10.5281/zenodo.23851128](https://doi.org/10.5281/zenodo.23851128)).
201
204
  Citation metadata: [CITATION.cff](CITATION.cff).
202
205
 
203
206
  ## What this does **not** prove
@@ -9,8 +9,8 @@ Project site: <https://idemcore.net>
9
9
  Idemcore is an integer-only inference engine: the same model, prompt, and
10
10
  settings give the *same bits* on an x86 laptop, an ARM phone, and NVIDIA GPUs
11
11
  from Turing to Hopper. Every token it scores or generates emits a 64-bit hash
12
- of the full logit vector and a 64-bit hash of the integer sampling table
13
- (CDF). A run log is a list of these hashes plus one chained digest (`ALL`).
12
+ of the full logit vector and a 64-bit hash of the integer token-selection
13
+ table. A run log is a list of these hashes plus one chained digest (`ALL`).
14
14
 
15
15
  This repository is the **open half** of Idemcore:
16
16
 
@@ -79,7 +79,8 @@ receipt to every answer, and `GET /v1/receipts/{id}/log` returns that answer's
79
79
  per-token log. The API serves the public form of the log (`log_format`
80
80
  `idemcore-log/r1-public`): the `pi j h hc` fields of [SPEC.md](SPEC.md) plus the
81
81
  `ALL` line, without `freq` and `total`. The verifier accepts both forms (version
82
- 0.3.0 and later). Access needs an API key: sign in to the console at
82
+ 0.3.0 and later; hash-only batched logs from 0.3.2). The sample logs in
83
+ `samples/` are published in the same hash-only form. Access needs an API key: sign in to the console at
83
84
  <https://console.idemcore.net/#/login> and create one. With a key, save the log and run:
84
85
 
85
86
  ```sh
@@ -111,8 +112,9 @@ Requires Python ≥ 3.9, standard library only.
111
112
 
112
113
  ## What the verifier checks
113
114
 
114
- 1. **Well-formedness** – every line has six fields, hashes are 64-bit hex,
115
- `0 < freq ≤ total < 2^31`, no duplicate positions.
115
+ 1. **Well-formedness** – every line has four fields (hash-only) or six,
116
+ hashes are 64-bit hex, `0 < freq ≤ total < 2^31` when present, no
117
+ duplicate positions.
116
118
  2. **Chain integrity** – it recomputes the `ALL` digest from the per-token
117
119
  lines; editing a single line breaks it.
118
120
  3. **Agreement** – it compares a candidate log against a reference log
@@ -122,7 +124,7 @@ Requires Python ≥ 3.9, standard library only.
122
124
 
123
125
  A match on `h` means the two machines computed the **same integer logits for
124
126
  the whole vocabulary** at that position, not just the same argmax token. A
125
- match on `hc` means they built the same integer sampling distribution, so any
127
+ match on `hc` means they built the same integer token-selection table, so any
126
128
  seeded sampler driven by it makes the same choices.
127
129
 
128
130
  ## Published pairs
@@ -167,7 +169,8 @@ The raw batched logs are in `samples/batched/` and are re-checked by step 3 abov
167
169
 
168
170
  Don Wick, *Idemcore: Bit-Identical LLM Inference from Phones to Datacenter
169
171
  GPUs, with an Open Per-Token Verification Protocol*, preprint, Zenodo,
170
- 2026. DOI: [10.5281/zenodo.23851129](https://doi.org/10.5281/zenodo.23851129).
172
+ 2026, version 1.1. DOI: [10.5281/zenodo.23949090](https://doi.org/10.5281/zenodo.23949090)
173
+ (all versions: [10.5281/zenodo.23851128](https://doi.org/10.5281/zenodo.23851128)).
171
174
  Citation metadata: [CITATION.cff](CITATION.cff).
172
175
 
173
176
  ## What this does **not** prove
@@ -2,4 +2,4 @@
2
2
 
3
3
  Pure Python standard library. No model, no kernel, no GPU needed.
4
4
  """
5
- __version__ = "0.3.0"
5
+ __version__ = "0.3.2"
@@ -8,17 +8,19 @@ A log is a text file with one line per generated/scored token:
8
8
  pi payload (sequence) index, decimal
9
9
  j token position inside the payload, decimal
10
10
  h FNV-1a-64 of the V int32 logits (little-endian), 16 hex digits
11
- hc FNV-1a-64 of the (V+1) uint64 integer CDF table (little-endian), 16 hex
12
- freq CDF width of the observed token: CUM[t+1] - CUM[t], decimal
13
- total CDF total CUM[V], decimal, 0 < total < 2**31
11
+ hc FNV-1a-64 of the engine's integer token-selection table (uint64, LE), 16 hex
12
+ freq opaque engine-defined integer for the observed token, decimal
13
+ total opaque engine-defined integer, decimal; 0 < freq <= total < 2**31
14
+ (freq/total are range-checked and compared, but are not part of ALL)
14
15
 
15
- The hosted API serves the *public* form of the same log
16
- (``idemcore-log/r1-public``): the first four fields only,
16
+ The *hash-only* (public) form of the same log (``idemcore-log/r1-public``,
17
+ served by the hosted API and used for the published sample logs) carries the
18
+ first four fields only,
17
19
 
18
20
  <pi> <j> <h> <hc>
19
21
 
20
22
  This verifier accepts both forms. ``ALL`` depends only on ``h`` and ``hc``, so
21
- it is the same for the full and the public form of a run.
23
+ it is the same for the full and the hash-only form of a run.
22
24
 
23
25
  followed by a chain line
24
26
 
@@ -89,7 +91,7 @@ class Log:
89
91
 
90
92
 
91
93
  def read_lines(path: str) -> List[str]:
92
- """Read a log as ASCII lines; files ending in .gz are decompressed."""
94
+ """Read a log as ASCII lines; files ending in .gz are read with gzip."""
93
95
  opener = gzip.open if path.endswith(".gz") else open
94
96
  with opener(path, "rt", encoding="ascii") as f:
95
97
  return f.read().splitlines()
@@ -115,7 +117,7 @@ def parse_log(path: str, lines: Optional[Iterable[str]] = None) -> Log:
115
117
  if not p[0].isdigit():
116
118
  continue # reserved keyword line (format v1 ignores it)
117
119
  if len(p) not in (4, 6):
118
- errors.append(f"line {n}: expected 6 fields (or 4 in a public log), got {len(p)}")
120
+ errors.append(f"line {n}: expected 6 fields (or 4 in a hash-only log), got {len(p)}")
119
121
  continue
120
122
  try:
121
123
  pi, j = int(p[0]), int(p[1])
@@ -215,7 +217,8 @@ def compare_logs(ref: Log, cand: Log, subset: bool = False) -> CompareResult:
215
217
  # ---------------------------------------------------------------------------
216
218
  # Batched (multi-stream) logs, format v1-batch. See SPEC.md, "Batched logs".
217
219
  #
218
- # <s> <pi> <j> <h> <hc> <freq> <total> one line per token of stream s
220
+ # <s> <pi> <j> <h> <hc> [<freq> <total>] one line per token of stream s
221
+ # (hash-only form: first 5 fields)
219
222
  # ALL <s> <16 hex> chain of stream s, in file order
220
223
  #
221
224
  # Stream ids >= FILLER_BASE mark filler work that occupied a slot before a
@@ -268,21 +271,23 @@ def parse_batch_log(path: str, lines: Optional[Iterable[str]] = None) -> BatchLo
268
271
  continue
269
272
  if not p[0].isdigit():
270
273
  continue # reserved keyword line
271
- if len(p) != 7:
272
- errors.append(f"line {n}: expected 7 fields, got {len(p)}")
274
+ if len(p) not in (5, 7):
275
+ errors.append(f"line {n}: expected 7 fields (or 5 in a hash-only log), got {len(p)}")
273
276
  continue
274
277
  try:
275
278
  sid, pi, j = int(p[0]), int(p[1]), int(p[2])
276
279
  if len(p[3]) != 16 or len(p[4]) != 16:
277
280
  raise ValueError("hash field is not 16 hex digits")
278
281
  h, hc = int(p[3], 16), int(p[4], 16)
279
- freq, total = int(p[5]), int(p[6])
282
+ freq = total = None
283
+ if len(p) == 7:
284
+ freq, total = int(p[5]), int(p[6])
280
285
  except ValueError as e:
281
286
  errors.append(f"line {n}: {e}")
282
287
  continue
283
- if not (0 < total < 2 ** 31):
288
+ if total is not None and not (0 < total < 2 ** 31):
284
289
  errors.append(f"line {n}: total {total} out of range (0, 2^31)")
285
- if not (0 < freq <= total):
290
+ if freq is not None and total is not None and not (0 < freq <= total):
286
291
  errors.append(f"line {n}: freq {freq} not in (0, total]")
287
292
  rows.append((sid, TokenLine(pi, j, h, hc, freq, total, n)))
288
293
  return BatchLog(path, rows, alls, errors)
@@ -1,11 +1,11 @@
1
1
  Metadata-Version: 2.4
2
2
  Name: idemcore-verify
3
- Version: 0.3.0
3
+ Version: 0.3.2
4
4
  Summary: Check Idemcore per-token hash logs: verify, without trusting us, that two machines produced bit-identical LLM inference.
5
5
  Author-email: Don Wick <don.wick@idemcore.net>
6
6
  License-Expression: Apache-2.0
7
7
  Project-URL: Homepage, https://idemcore.net
8
- Project-URL: Paper, https://doi.org/10.5281/zenodo.23851129
8
+ Project-URL: Paper, https://doi.org/10.5281/zenodo.23851128
9
9
  Project-URL: Source, https://github.com/idemcore/idemcore-verifier
10
10
  Project-URL: Issues, https://github.com/idemcore/idemcore-verifier/issues
11
11
  Keywords: reproducibility,deterministic inference,llm,verification,integer inference
@@ -39,8 +39,8 @@ Project site: <https://idemcore.net>
39
39
  Idemcore is an integer-only inference engine: the same model, prompt, and
40
40
  settings give the *same bits* on an x86 laptop, an ARM phone, and NVIDIA GPUs
41
41
  from Turing to Hopper. Every token it scores or generates emits a 64-bit hash
42
- of the full logit vector and a 64-bit hash of the integer sampling table
43
- (CDF). A run log is a list of these hashes plus one chained digest (`ALL`).
42
+ of the full logit vector and a 64-bit hash of the integer token-selection
43
+ table. A run log is a list of these hashes plus one chained digest (`ALL`).
44
44
 
45
45
  This repository is the **open half** of Idemcore:
46
46
 
@@ -109,7 +109,8 @@ receipt to every answer, and `GET /v1/receipts/{id}/log` returns that answer's
109
109
  per-token log. The API serves the public form of the log (`log_format`
110
110
  `idemcore-log/r1-public`): the `pi j h hc` fields of [SPEC.md](SPEC.md) plus the
111
111
  `ALL` line, without `freq` and `total`. The verifier accepts both forms (version
112
- 0.3.0 and later). Access needs an API key: sign in to the console at
112
+ 0.3.0 and later; hash-only batched logs from 0.3.2). The sample logs in
113
+ `samples/` are published in the same hash-only form. Access needs an API key: sign in to the console at
113
114
  <https://console.idemcore.net/#/login> and create one. With a key, save the log and run:
114
115
 
115
116
  ```sh
@@ -141,8 +142,9 @@ Requires Python ≥ 3.9, standard library only.
141
142
 
142
143
  ## What the verifier checks
143
144
 
144
- 1. **Well-formedness** – every line has six fields, hashes are 64-bit hex,
145
- `0 < freq ≤ total < 2^31`, no duplicate positions.
145
+ 1. **Well-formedness** – every line has four fields (hash-only) or six,
146
+ hashes are 64-bit hex, `0 < freq ≤ total < 2^31` when present, no
147
+ duplicate positions.
146
148
  2. **Chain integrity** – it recomputes the `ALL` digest from the per-token
147
149
  lines; editing a single line breaks it.
148
150
  3. **Agreement** – it compares a candidate log against a reference log
@@ -152,7 +154,7 @@ Requires Python ≥ 3.9, standard library only.
152
154
 
153
155
  A match on `h` means the two machines computed the **same integer logits for
154
156
  the whole vocabulary** at that position, not just the same argmax token. A
155
- match on `hc` means they built the same integer sampling distribution, so any
157
+ match on `hc` means they built the same integer token-selection table, so any
156
158
  seeded sampler driven by it makes the same choices.
157
159
 
158
160
  ## Published pairs
@@ -197,7 +199,8 @@ The raw batched logs are in `samples/batched/` and are re-checked by step 3 abov
197
199
 
198
200
  Don Wick, *Idemcore: Bit-Identical LLM Inference from Phones to Datacenter
199
201
  GPUs, with an Open Per-Token Verification Protocol*, preprint, Zenodo,
200
- 2026. DOI: [10.5281/zenodo.23851129](https://doi.org/10.5281/zenodo.23851129).
202
+ 2026, version 1.1. DOI: [10.5281/zenodo.23949090](https://doi.org/10.5281/zenodo.23949090)
203
+ (all versions: [10.5281/zenodo.23851128](https://doi.org/10.5281/zenodo.23851128)).
201
204
  Citation metadata: [CITATION.cff](CITATION.cff).
202
205
 
203
206
  ## What this does **not** prove
@@ -32,7 +32,7 @@ dependencies = []
32
32
 
33
33
  [project.urls]
34
34
  Homepage = "https://idemcore.net"
35
- Paper = "https://doi.org/10.5281/zenodo.23851129"
35
+ Paper = "https://doi.org/10.5281/zenodo.23851128"
36
36
  Source = "https://github.com/idemcore/idemcore-verifier"
37
37
  Issues = "https://github.com/idemcore/idemcore-verifier/issues"
38
38
 
@@ -59,6 +59,23 @@ class T(unittest.TestCase):
59
59
  five = pub[:100] + [pub[100] + " 7"] + pub[101:]
60
60
  self.assertFalse(check_log(parse_log("five", five)).ok)
61
61
 
62
+ def test_legacy_six_field_log(self):
63
+ # Logs written before 0.3.2 carry two extra integers per line; they
64
+ # are range-checked and compared but are not part of ALL.
65
+ p = os.path.join(S, "qwen2.5-7b-int4-g", "x86_64.log")
66
+ with open(p) as fh:
67
+ lines = fh.read().splitlines()
68
+ six = [l + " 3 7" if l[:1].isdigit() else l for l in lines]
69
+ log6 = parse_log("six", six)
70
+ r = check_log(log6)
71
+ self.assertTrue(r.ok, r.errors)
72
+ self.assertEqual(r.all_recomputed, check_log(parse_log(p)).all_recomputed)
73
+ self.assertTrue(compare_logs(parse_log(p), log6).ok) # hash-only vs 6-field
74
+ other = [l + " 4 7" if l[:1].isdigit() else l for l in lines]
75
+ self.assertFalse(compare_logs(log6, parse_log("other", other)).ok)
76
+ bad = six[:100] + [" ".join(six[100].split()[:4] + ["0", "7"])] + six[101:]
77
+ self.assertFalse(check_log(parse_log("bad", bad)).ok) # range check
78
+
62
79
  if __name__ == "__main__":
63
80
  unittest.main()
64
81
 
@@ -95,6 +112,20 @@ class BatchTests(unittest.TestCase):
95
112
  finally:
96
113
  os.unlink(t.name)
97
114
 
115
+ def test_batch_hash_only_and_legacy(self):
116
+ from idemcore_verify.core import check_batch_log, compare_batch, parse_batch_log, read_lines
117
+ lines = read_lines(self.B)
118
+ five = parse_batch_log("five", lines)
119
+ self.assertTrue(all(t.freq is None for _, t in five.rows))
120
+ seven = [l + " 3 7" if l[:1].isdigit() else l for l in lines]
121
+ b7 = parse_batch_log("seven", seven)
122
+ c5, c7 = check_batch_log(five), check_batch_log(b7)
123
+ self.assertTrue(c7.ok, c7.errors)
124
+ self.assertEqual(c5.streams_digest, c7.streams_digest)
125
+ self.assertTrue(compare_batch(parse_log(self.REF), b7).ok)
126
+ six = [l + " 3" if l[:1].isdigit() else l for l in lines[:50]] + lines[50:]
127
+ self.assertFalse(check_batch_log(parse_batch_log("six", six)).ok)
128
+
98
129
  def test_self_check_passes(self):
99
130
  import contextlib, io
100
131
  from idemcore_verify.samples import run_samples
File without changes