ckb-cycle-tools 1.2.0 → 1.3.0

This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
package/CHANGELOG.md CHANGED
@@ -2,6 +2,124 @@
2
2
 
3
3
  This project follows [semantic versioning](https://semver.org/).
4
4
 
5
+ ## 1.3.0 (2026-11-07)
6
+
7
+ From the review on #46 (5 Nov): failed gates on a script that starts rejecting,
8
+ a reproducibility manifest, and what a rejected transaction's numbers mean.
9
+
10
+ ### Changed
11
+ - `diff --fail-above` (and the Action's `fail-above`) now also fails when a
12
+ group that accepted on the base rejects on the head (`ok` → `exit N` or a VM
13
+ error), whatever its cycles: a script that starts rejecting what it accepted
14
+ could pass before, by getting cheaper. This affects workflows that set only
15
+ `fail-above`: a pull request that turns an accepting scenario into a
16
+ rejection now fails the check (a warning with `informational: true`).
17
+ `--fail-on-result-change` is unchanged and still broader (reject → accept,
18
+ one exit code to another, a new group that does not accept); a diff with
19
+ neither set stays informational. The comment marks such a row "accepted on
20
+ the base, rejects now".
21
+ - A gated diff (`--fail-above` or `--fail-on-result-change`) now also fails
22
+ when a scenario's inputs changed between base and head (unless
23
+ `--allow-changed-inputs`), or when the two sides ran another ckb-debugger or
24
+ CKB-VM version. Before, an edited mock under the same scenario name was
25
+ compared silently. Ungated diffs only gain notes. See Added.
26
+ - A gated diff also fails when a head scenario's groups add up to more than
27
+ the node's per-transaction cycle limit and the base's did not: the node would
28
+ reject it, as if a group started rejecting (see Added). A new scenario over
29
+ the limit fails `--fail-on-result-change` only. The CLI exits 1 for such a
30
+ transaction.
31
+
32
+ ### Added
33
+ - `report` writes a `manifest` into its JSON: the ckb-cycle-tools version,
34
+ ckb-debugger's `--version` and the sha256 (and path) of the binary every
35
+ scenario ran (resolved once), or a WebAssembly build's id for a library
36
+ caller that runs one (`debuggerBuild`), the git commit of the config's
37
+ checkout and of `--binaries-from` (none, and no failure, outside git), an
38
+ optional `--toolchain <text>`, and per scenario the CKB-VM version, an
39
+ `inputs` sha256 (the mock or transaction file's bytes as read, and the
40
+ cycles.json entry canonicalised without its name, file path or `functions`)
41
+ and each swapped-in binary's path, sha256 and size. The terminal report
42
+ ends with a manifest line.
43
+ - `diff` reads both manifests. A scenario whose `inputs` differ is NOT
44
+ COMPARED (its groups count as neither new nor removed); a gated diff fails
45
+ on it unless `--allow-changed-inputs`. Another ckb-debugger (version, binary,
46
+ native against WebAssembly) or another CKB-VM version for a scenario fails a
47
+ gated diff, listed under the summary; ungated it is a bold note. Another
48
+ toolchain is a note. A report without a manifest on one side is a note,
49
+ never a failure; two such reports diff as before. The comment and the
50
+ terminal output show each side's manifest in one line. Manifest strings go
51
+ through the same cleaning as other report text.
52
+ - Action inputs `toolchain` (a command run after the build in each checkout;
53
+ its output goes to `--toolchain`) and `allow-changed-inputs`.
54
+ - A rejected transaction (some group exits non-zero, the VM stops it, or Type
55
+ ID refuses it) no longer shows "measured (sum of groups)". The CLI prints
56
+ "sum of groups, each run on its own", names the group the node would stop
57
+ at, says the node reports no cycles for a rejected transaction, and lists the
58
+ groups the node would never run. The order is ckb-script's, read from its
59
+ source (`TransactionScriptsVerifier::verify` and `TxData`/`TxInfo::groups`
60
+ at ckb a4aa128): lock groups, then type groups, each by script hash,
61
+ stopping at the first failure.
62
+ - JSON, added fields only: a profile (`--json`, `profileTransaction`) and a
63
+ report scenario get `nodeStopsAt` (that group's selector), and each group
64
+ `nodeReaches: true | false`. Both are only set when some group rejects, so
65
+ reports of accepted transactions are byte-for-byte as before. `report`'s text
66
+ output says the same under a rejected scenario. `diff` and its gates are
67
+ unchanged.
68
+ - The node's cycle limit is per transaction (`max_block_cycles`, 3,500,000,000
69
+ on mainnet and testnet: each group gets what the ones before it left), while
70
+ ckb-debugger gives each group 3,500,000,000 on its own. A transaction whose
71
+ groups each fit but together pass the limit was shown as accepted; it is now
72
+ flagged: "over the node's per-transaction limit of 3,500,000,000 cycles: the
73
+ node would reject this transaction (exceeded max cycles)", in the CLI table,
74
+ the `report` text and the `diff` summary and comment. The limit is the
75
+ node's `get_consensus` `max_block_cycles` when the replay asked the node,
76
+ else 3,500,000,000 (`txCycleLimit` in the options to override;
77
+ `DEFAULT_TX_CYCLE_LIMIT` exported). `nodeStopsAt` and `nodeReaches` then mark
78
+ the group at which the running sum, in the node's order, passes the limit,
79
+ unless a failing group comes first. JSON, added fields only, and only in
80
+ that case: `overTxLimit: { limit, sum }` on a profile (bigints as strings,
81
+ as `measuredCycles`) and on a report scenario (numbers), and in a diff, the
82
+ head scenarios newly over it. Transactions under the limit are unchanged.
83
+ - `nodeOrder`, `nodeStop` and `sumLines` exported from the package and from
84
+ `ckb-cycle-tools/replay` (no Node built-ins).
85
+
86
+ ### Docs
87
+ - compat/HARDFORK.md, from the review on #46 (point 1): 24 real mainnet and
88
+ testnet transactions next to Mirana (VM v1) and Meepo (VM v2), the nearest
89
+ of each kind (secp only; another `type` script; a data/data1/data2 script)
90
+ on each side of each fork, replayed as `ckb-cycles <hash>` replays them and
91
+ again with `--script-version` forced to the two other values. The 18 after
92
+ Mirana match the cycles a node recorded for their block exactly, at the
93
+ chosen version only, and data/data1/data2 groups keep their own version.
94
+ Before Mirana no public node has cycles (mainnet.ckb.dev reports `0x0`
95
+ everywhere here; mainnet/testnet.ckbapp.dev record from the Mirana block on),
96
+ and the explorer's number for those 6 equals the v1 sum, not the v0 sum the
97
+ consensus rules give: reported as unverified, with how to settle it. Also why
98
+ `estimate_cycles` is no oracle for a historical transaction (spent inputs do
99
+ not resolve; it would run today's VM).
100
+ - `npm run hardfork` (scripts/hardfork-check.ts, `--discover` to pick the
101
+ transactions again) writes compat/hardfork.json, the table and
102
+ test/fixtures/hardfork-mocks.json.br. test/hardfork.test.ts replays the
103
+ recorded mocks offline with the WebAssembly debugger at every version, and
104
+ with `CKB_HARDFORK_NETWORK=1` plans them again through the public nodes.
105
+ No change to the replay itself.
106
+ - README: "What a rejected transaction's numbers mean", and "What comes from
107
+ where when replaying": a table of every input to a replay (cells, cell deps
108
+ and dep groups, input headers, header deps, consensus epochs, CKB-VM version,
109
+ max cycles, group order, Type ID, since/capacity checks) and whether it comes
110
+ from the chain via RPC, the node's current state or ckb-debugger's defaults;
111
+ ckb-debugger's limit is per group, the node's per transaction.
112
+ - README: "Flamegraph fidelity": totals are exact, the split between functions
113
+ is attributed (frames open only at symbol-table function starts, so inlined
114
+ code is charged to its caller; cycles charged outside instructions are in no
115
+ frame). The flame example now says the 1,812-cycle gap is everything charged
116
+ outside instructions, not only loading the binary.
117
+ - benchmarks/session-lock/RESULTS.md (and compare.mjs, which writes it): the
118
+ comparison is of these two implementations only, and the Rust lock's build
119
+ profile at 49227cd (`opt-level = "s"`, LTO, one codegen unit, `panic = "abort"`,
120
+ overflow checks, stripped; `+zba,+zbb,+zbc,+zbs,-a`; ckb-std 1.1.0; no pinned
121
+ toolchain).
122
+
5
123
  ## 1.2.0 (2026-11-05)
6
124
 
7
125
  Scripts that spawn other scripts (CKB2023 `spawn`).
package/README.md CHANGED
@@ -44,7 +44,7 @@ npm install -g ckb-cycle-tools # the ckb-cycles command
44
44
  ckb-cycles <tx hash> # or without installing: npx -p ckb-cycle-tools ckb-cycles <tx hash>
45
45
  ```
46
46
 
47
- [ckb-cycle-tools on npm](https://www.npmjs.com/package/ckb-cycle-tools) (v1.2.0, ESM, Node 20 or
47
+ [ckb-cycle-tools on npm](https://www.npmjs.com/package/ckb-cycle-tools) (v1.3.0, ESM, Node 20 or
48
48
  later). It drives `ckb-debugger`, which must be on PATH or passed with `--debugger`.
49
49
 
50
50
  ## Try it
@@ -178,14 +178,75 @@ tx not on chain
178
178
  GROUP SCRIPT RUN AT RESULT CYCLES SHARE
179
179
  lock 0x1ae8b8…26bc20 input.0.lock exit 12 12,893 0.8%
180
180
  lock Secp256k1Blake160 input.1.lock exit -31 1,642,194 99.2%
181
+
182
+ sum of groups, each run on its own 1,655,087
183
+ not what the node spends: it stops at the first failing group, input.1.lock (Secp256k1Blake160),
184
+ and reports no cycles for a rejected tx (it runs lock groups, then type groups,
185
+ each in script-hash order)
186
+ the node never runs: input.0.lock
181
187
  ```
182
188
 
183
189
  The session lock refuses with its code 12 (outflow over `max_per_tx`) after
184
190
  12,893 cycles; the secp256k1 lock fails too, because the edit invalidated the
185
191
  signature. CI replays this file and requires exactly that refusal.
186
192
 
193
+ **What a rejected transaction's numbers mean.** The CLI runs every group on its
194
+ own, so you see every script that refuses, not only the first. The node does not
195
+ work that way. ckb-script's `TransactionScriptsVerifier::verify` runs the groups
196
+ one after another and returns at the first error, with no cycle count; a
197
+ rejected transaction has no cycles on the node at all. Its order is every lock
198
+ group, then every type group, each sorted by script hash (two `BTreeMap`s keyed by
199
+ script hash, locks chained before types), which is not the order the CLI lists
200
+ them in (first appearance). So for a rejected transaction the sum is labelled as
201
+ what it is, and the CLI names the group the node would stop at and the groups it
202
+ would never run. Above, the secp256k1 lock's script hash sorts first, so the node
203
+ rejects on the signature and never runs the session lock. In `--json` and in a
204
+ report's JSON, the scenario or profile gets `nodeStopsAt` (that group's selector)
205
+ and every group gets `nodeReaches: true | false`; both are absent when every
206
+ group accepts, so reports of accepted transactions are unchanged. The same
207
+ applies to a transaction whose groups each accept but together pass the node's
208
+ per-transaction cycle limit (see Max cycles below): it gets `overTxLimit:
209
+ { limit, sum }` too, and `nodeStopsAt` names the group at which the running
210
+ sum, in the node's order, passes the limit. Sources, from
211
+ ckb `develop` at
212
+ [a4aa128](https://github.com/nervosnetwork/ckb/tree/a4aa1286a45bc1d42c30f6698dc51521c3749b40):
213
+ [`script/src/verify.rs`](https://github.com/nervosnetwork/ckb/blob/a4aa1286a45bc1d42c30f6698dc51521c3749b40/script/src/verify.rs#L180-L197)
214
+ (`verify`, stops at the first failing group) and
215
+ [`script/src/types.rs`](https://github.com/nervosnetwork/ckb/blob/a4aa1286a45bc1d42c30f6698dc51521c3749b40/script/src/types.rs#L642-L666)
216
+ with [L865-L868](https://github.com/nervosnetwork/ckb/blob/a4aa1286a45bc1d42c30f6698dc51521c3749b40/script/src/types.rs#L865-L868)
217
+ (group order). Which group fails first is computed, not observed: no node has
218
+ been asked to confirm it for a rejected transaction yet.
219
+
187
220
  In code: `profileTransaction(readTransactionJson(json))`, or pass an RPC-shaped
188
- transaction object directly.
221
+ transaction object directly; `nodeOrder(groups)` and `nodeStop(groups)` give the
222
+ node's order and where it stops.
223
+
224
+ ### What comes from where when replaying
225
+
226
+ A replay rebuilds everything a script can observe from the chain, and takes the
227
+ rest from the node it asks or from ckb-debugger's defaults:
228
+
229
+ | Input to the replay | Where it comes from |
230
+ |---|---|
231
+ | Input cells (output, data) | The original chain via RPC: `get_transaction` of the transaction that created each cell, so a spent cell replays as it was, not from the live cell set |
232
+ | Cell deps, dep groups | Same, from the creating transactions; a dep group's cell is read and every cell it lists added as a code dep |
233
+ | Input cell headers | The block that created each input cell (its `block_hash`), as on the node; a script sees it only if the transaction lists it in `header_deps`, as on the node |
234
+ | Header deps | `get_header` for each hash in the transaction's `header_deps` |
235
+ | Witnesses, outputs, outputs data | The transaction itself (from `get_transaction`, or the `--file` you pass) |
236
+ | Hardfork epochs (consensus) | The node's current `get_consensus` (RFC 0032 for CKB-VM v1, RFC 0049 for v2); a built-in table only when the public node of `--mainnet`/testnet cannot say |
237
+ | CKB-VM version | For `hash_type: "type"` scripts: the version at the epoch of the transaction's block, from `get_header`, on those consensus epochs. `data`, `data1`, `data2` name their own. A transaction not on chain (`--file`) gets v2, today's version, not one read from the node |
238
+ | Max cycles | ckb-debugger's default, 3,500,000,000 **per group** (each group is its own run). The node's limit is **per transaction**: in a block, each transaction is verified with the consensus `max_block_cycles` (3,500,000,000 on mainnet and testnet), the remaining budget shrinking group by group, and the block's sum must stay under it too. So a transaction whose groups each fit but together exceed the limit passes ckb-debugger and fails on the node; the CLI, `report` and `diff` flag it ("over the node's per-transaction limit ...: the node would reject this transaction (exceeded max cycles)", `overTxLimit` in the JSON, exit 1), with the node's `max_block_cycles` from `get_consensus` when the replay asked the node, else 3,500,000,000. A gated `diff` fails on a scenario newly over it |
239
+ | Group order | Here: first appearance, every group run, failures included. Node: lock groups then type groups, each by script hash, stopping at the first failure (see above) |
240
+ | Type ID | Not a binary: checked here as the node checks it, and charged the node's fixed `TYPE_ID_CYCLES` (1,000,000) |
241
+ | Since, capacity, maturity checks | Not replayed: the node checks them outside any script, so a script-level replay cannot see them |
242
+ | Script names | CCC's known-script table for the network, and your `--name`s; labels only |
243
+
244
+ The node's own verification lives in
245
+ [`verification/contextual/src/contextual_block_verifier.rs`](https://github.com/nervosnetwork/ckb/blob/a4aa1286a45bc1d42c30f6698dc51521c3749b40/verification/contextual/src/contextual_block_verifier.rs#L413-L466)
246
+ (per-transaction limit and block sum) and ckb-debugger's default in
247
+ [`ckb-debugger/src/main.rs`](https://github.com/nervosnetwork/ckb-standalone-debugger/blob/740d6de0c1d7c2394a492f9bfef57e77abb98dfd/ckb-debugger/src/main.rs#L44)
248
+ of v1.1.1. A node's transaction pool also has a `max_tx_verify_cycles` setting
249
+ in its `ckb.toml`: a local setting, not consensus.
189
250
 
190
251
  ### Inside a script: flamegraphs
191
252
 
@@ -219,8 +280,9 @@ input.0.lock 0xab4194…4c08ce ok 20,999 cycles
219
280
  ```
220
281
 
221
282
  Reading the lock hashes of the transaction's cells is the lock's largest cost.
222
- The profile covers the script's run; loading the binary into the VM (1,812
223
- cycles here) is counted by the node but not by the profiler.
283
+ The profile covers the instructions the script ran; the difference (1,812 cycles
284
+ here) is what the VM charges outside instructions, such as loading the binary and
285
+ the bytes each syscall copies, counted by the node but in no frame.
224
286
 
225
287
  **A script that spawns another** (CKB2023 `spawn`): its group's cycles include
226
288
  every script it spawned, as on the node, and the CLI and report say how many it
@@ -229,6 +291,32 @@ that one, and `--bin` replaces only its binary, so the spawned scripts' work is
229
291
  in the total but in no frame. [`test/fixtures/spawn-project`](test/fixtures/spawn-project)
230
292
  is such a pair.
231
293
 
294
+ **Flamegraph fidelity: what is exact and what is attributed.** The group's cycle
295
+ total is exact: it is ckb-debugger's run of the deployed binary, the same number
296
+ the node counts (and `--bin` checks that the symbols build costs the same). How
297
+ that total is split between functions is reconstructed. ckb-debugger's profiler
298
+ (`machine_analyzer.rs` in
299
+ [v1.1.1](https://github.com/nervosnetwork/ckb-standalone-debugger/blob/740d6de0c1d7c2394a492f9bfef57e77abb98dfd/ckb-debugger/src/machine_analyzer.rs))
300
+ charges each instruction's cost (ckb-vm's cost model) to the current frame, and
301
+ opens a frame only when a `jal`/`jalr` jumps to an address the ELF symbol table
302
+ lists as a function start. So:
303
+
304
+ - inlined code has no symbol of its own and is charged to the function it was
305
+ inlined into, so an optimised build shows fewer, heavier frames than the source
306
+ has functions, and moving code between inlined and not changes the split
307
+ without changing the total;
308
+ - frames are named with the binary's DWARF info where it has some, and otherwise
309
+ from the symbol table (this tool fills in the `func_0x…` frames the profiler
310
+ could not name);
311
+ - cycles the VM charges outside instructions (loading the binary, bytes copied by
312
+ syscalls) are in no frame, so the profile is a little under the group total and
313
+ the CLI prints both;
314
+ - spawned scripts are in the total but in no frame (above).
315
+
316
+ Per-function numbers are therefore exact sums of the instructions attributed to
317
+ each frame, but which frame gets them depends on the build's inlining and symbols.
318
+ Compare functions only between builds with the same profile.
319
+
232
320
  **In the browser:** drop the `.folded` file on the
233
321
  [web inspector](https://ckb-cycle-tools.vercel.app/) ("inside a script"), or open
234
322
  its example, and it draws the same flamegraph and table. Nothing is uploaded.
@@ -267,13 +355,43 @@ the head to compare against (a scenario or group removed, renamed or moved, or
267
355
  measured without its binary); `--allow-removed` accepts the last. A group the VM
268
356
  stopped (past the cycle limit, a bad instruction) is a result change, and over
269
357
  any `--fail-above` threshold, never a -100% saving; two different VM errors are
270
- two different results. With `--binaries-from`, a binary missing from that
358
+ two different results. A group that accepted on the base and rejects on the
359
+ head (`ok` → `exit 5`, or a VM error) fails `--fail-above` too, whatever its
360
+ cycles: a cheaper rejection is not a saving. `--fail-on-result-change` is
361
+ broader: it also fails on a rejection that now accepts, one exit code turning
362
+ into another, and a new group that does not accept. With `--binaries-from`, a binary missing from that
271
363
  checkout is left as the mock has it: only the groups whose code it is are not
272
364
  compared, and every other binary is still swapped in. A binary offered to a
273
365
  scenario that matches none of its cell deps is a warning on that scenario, in
274
366
  the comment too. The summary says when the two reports come from different
275
367
  ckb-debugger versions.
276
368
 
369
+ #### What a report was measured with: the manifest
370
+
371
+ Each report's JSON has a `manifest` (from 1.3.0 on): the ckb-cycle-tools version,
372
+ ckb-debugger's `--version` and the sha256 of the binary every scenario ran (or a
373
+ WebAssembly build's id, when a library caller runs one), the git commit of the
374
+ checkout the cycles.json is in (and of `--binaries-from`; none outside a git
375
+ checkout), an optional `--toolchain <text>` (say, `rustc -V` and the build
376
+ profile), and per scenario the CKB-VM version, an `inputs` sha256 and each
377
+ swapped-in binary's path, sha256 and size. `inputs` covers what decides what a
378
+ scenario runs: the bytes of its mock or transaction file as read, and its
379
+ cycles.json entry without its name or file path (its form, transaction hash,
380
+ `binaries` list, `scriptVersion`, and the network for a transaction fetched
381
+ from a node).
382
+
383
+ `diff` reads both manifests. A scenario whose `inputs` differ between base and
384
+ head did not measure the same transaction: its groups are **NOT COMPARED:
385
+ inputs changed**, and a gated diff (`--fail-above` or `--fail-on-result-change`)
386
+ fails on it unless `--allow-changed-inputs`. Another ckb-debugger (version,
387
+ binary, or native against WebAssembly) or another CKB-VM version for a scenario
388
+ fails a gated diff with the reason listed under the summary, and is a
389
+ prominent note otherwise. A different toolchain is a note. A report without a
390
+ manifest (written before 1.3.0) on one side is a note, never a failure; two
391
+ such reports diff as they always did. Every manifest string (a toolchain, a
392
+ path) is cleaned like any other report text before it reaches a terminal or a
393
+ comment.
394
+
277
395
  Here the ckb-js-vm session lock was changed to allow one shannon more than its
278
396
  limit and to do extra work per recipient check:
279
397
 
@@ -597,8 +715,19 @@ the Linux builds weekly and on demand; its first run on GitHub (2 Nov 2026) pass
597
715
  functions only with an unstripped build passed by `--bin` (or `symbols` in a
598
716
  cycles.json). The CLI draws the graphs, the PR comment lists the changed
599
717
  functions, and the web page draws a `.folded` file the CLI made.
718
+ - **Groups run on their own.** Every group is a separate ckb-debugger run with
719
+ its own 3,500,000,000-cycle limit; the node runs them in its own order under
720
+ one per-transaction limit and stops at the first failure. A rejected
721
+ transaction's sum is labelled as such, and the group the node would stop at
722
+ is computed from ckb-script's order, not yet confirmed by a node. A sum over
723
+ the per-transaction limit is flagged as a rejection, from the summed group
724
+ cycles; no node has been asked to confirm one yet
725
+ ([What comes from where](#what-comes-from-where-when-replaying)).
600
726
  - **Older blocks.** Some public nodes report `0x0` cycles for old blocks; the
601
- tools then say the total is unknown instead of claiming a match.
727
+ tools then say the total is unknown instead of claiming a match. Replays on
728
+ both sides of each VM hardfork, checked against recorded totals where a node
729
+ has them, and why nothing could be checked against a node before Mirana:
730
+ [compat/HARDFORK.md](compat/HARDFORK.md).
602
731
  - **The PR Action has run on one public pull request so far**, in this repository:
603
732
  [#1](https://github.com/luongthanhtung03/ckb-cycle-tools/pull/1) slows one check of
604
733
  the ckb-js-vm benchmark lock on purpose, and the comment reports it (+2.94% on that
package/dist/cli.js CHANGED
@@ -6,7 +6,7 @@ import { diffFailed, diffReports, formatMarkdown, formatText, termLine, termLine
6
6
  import { buildMockTransaction, httpRpc, mockFromTransaction } from "./mock.js";
7
7
  import { flameTree, foldedTotal, formatFolded, functionCosts, renderFlameSvg } from "./flame.js";
8
8
  import { flameTransaction, parseNames, profileTransaction, readTransactionJson } from "./profile.js";
9
- import { PUBLIC_RPC } from "./replay.js";
9
+ import { PUBLIC_RPC, sumLines } from "./replay.js";
10
10
  import { formatReport, runReport } from "./report.js";
11
11
  import { formatScan, scanTestMocks } from "./scan.js";
12
12
  const USAGE = `usage: ckb-cycles <tx-hash> [--mainnet] [--rpc <url>] [--timeout <s>] [--debugger <path>]
@@ -14,8 +14,9 @@ const USAGE = `usage: ckb-cycles <tx-hash> [--mainnet] [--rpc <url>] [--timeout
14
14
  ckb-cycles --file <tx.json> [same options]
15
15
  ckb-cycles report <cycles.json> [--json <out.json>] [--binaries-from <dir>]
16
16
  [--root <dir> ...] [--rpc <url>] [--timeout <s>] [--debugger <path>]
17
+ [--toolchain <text>]
17
18
  ckb-cycles diff <base.json> <head.json> [--markdown] [--fail-above <percent>]
18
- [--fail-on-result-change] [--allow-removed] [--title <s>]
19
+ [--fail-on-result-change] [--allow-removed] [--allow-changed-inputs] [--title <s>]
19
20
  [--base-label <s>] [--head-label <s>]
20
21
  ckb-cycles mock <tx-hash> | --file <tx.json> [-o <mock.json>] [--mainnet] [--rpc <url>]
21
22
  [--timeout <s>]
@@ -38,7 +39,8 @@ the cycles each group cost.
38
39
  repeat for several. Without :<hash-type> it applies to all of them
39
40
 
40
41
  Exit codes: 0 every script accepts (and the sum matches the node's)
41
- 1 a script rejects, or the sum differs from the node's
42
+ 1 a script rejects, the sum differs from the node's, or the sum is
43
+ over the node's per-transaction limit (max_block_cycles)
42
44
  2 usage or environment error: bad arguments, node unreachable,
43
45
  transaction not found, ckb-debugger missing or unreadable output
44
46
 
@@ -55,11 +57,18 @@ Exit codes: 0 every script accepts (and the sum matches the node's)
55
57
  cell dep in any scenario (both also in the JSON's "errors"):
56
58
  a rejection is a result, not an error. A binary missing from
57
59
  --binaries-from is left as the mock has it: only the groups
58
- whose code it is are marked, and not compared
60
+ whose code it is are marked, and not compared.
61
+ The JSON's "manifest" records what the numbers were measured
62
+ with: tool version, ckb-debugger version and the sha256 of its
63
+ binary, the git commit, per scenario the CKB-VM version, a hash
64
+ of its inputs and the swapped-in binaries' hashes. --toolchain
65
+ adds free text to it (e.g. the output of rustc -V)
59
66
  diff compares two reports group by group: Δ cycles, Δ%, and any group
60
67
  whose result changed (ok → exit 12, or back). --fail-above exits 1
61
68
  when a group grows by more than that percentage (a new group
62
69
  has no base, so no percentage: it is never over the threshold),
70
+ or a group that accepted on the base rejects on the head
71
+ (whatever its cycles; also with --fail-on-result-change),
63
72
  --fail-on-result-change when a result changed, including a new
64
73
  group that does not accept, or a group the VM stopped (also
65
74
  over any --fail-above). A scenario the head could not run
@@ -67,7 +76,12 @@ Exit codes: 0 every script accepts (and the sum matches the node's)
67
76
  unchecked: a scenario the base could not run, an error in
68
77
  either report, a base group with nothing in the head to
69
78
  compare with (removed, renamed, moved, or its binary not built
70
- on the base); --allow-removed accepts the last
79
+ on the base); --allow-removed accepts the last. With both
80
+ reports' manifests: a scenario whose inputs (mock, transaction
81
+ file, cycles.json entry) changed is NOT COMPARED, and fails a
82
+ gated diff unless --allow-changed-inputs; another ckb-debugger
83
+ (version or binary) or CKB-VM version fails a gated diff, and
84
+ is a note otherwise. A report without a manifest is a note
71
85
  mock saves a transaction as a ckb-debugger mock transaction, every cell
72
86
  it touches included, so a scenario can replay it with no node
73
87
  scan writes a cycles.json from the mock transactions a test suite wrote
@@ -101,7 +115,7 @@ function table(p) {
101
115
  const line = (cells) => cells.map((c, i) => (i >= 4 ? c.padStart(widths[i]) : c.padEnd(widths[i]))).join(" ");
102
116
  const title = p.nodeCycles === undefined && p.status === "not on chain" ? "tx not on chain" : `tx ${p.hash} (${p.status})`;
103
117
  const out = [title, "", line(head), ...rows.map(line), ""];
104
- out.push(`measured (sum of groups) ${fmt(p.measuredCycles)}`);
118
+ out.push(...sumLines(p, termLine));
105
119
  if (p.nodeCycles !== undefined) {
106
120
  const match = p.nodeCycles === p.measuredCycles;
107
121
  out.push(`reported by node ${fmt(p.nodeCycles)} ${match ? "✓ match" : "✗ MISMATCH"}`);
@@ -138,6 +152,7 @@ async function reportCommand(args) {
138
152
  debugger: { type: "string" },
139
153
  "binaries-from": { type: "string" },
140
154
  root: { type: "string", multiple: true },
155
+ toolchain: { type: "string" },
141
156
  },
142
157
  });
143
158
  if (positionals.length !== 1) {
@@ -150,6 +165,7 @@ async function reportCommand(args) {
150
165
  ...(values.debugger && { debuggerPath: values.debugger }),
151
166
  ...(values["binaries-from"] && { binariesRoot: values["binaries-from"] }),
152
167
  ...(values.root && { roots: values.root }),
168
+ ...(values.toolchain && { toolchain: values.toolchain }),
153
169
  onScenario: (s) => console.error(`${s.error ? "✗" : "·"} ${termLine(s.name)}`),
154
170
  });
155
171
  const { errors, notices, ...data } = report;
@@ -179,6 +195,7 @@ async function diffCommand(args) {
179
195
  "fail-above": { type: "string" },
180
196
  "fail-on-result-change": { type: "boolean" },
181
197
  "allow-removed": { type: "boolean" },
198
+ "allow-changed-inputs": { type: "boolean" },
182
199
  "base-label": { type: "string" },
183
200
  "head-label": { type: "string" },
184
201
  title: { type: "string" },
@@ -197,6 +214,7 @@ async function diffCommand(args) {
197
214
  ...(failAbove !== undefined && { failAbove }),
198
215
  failOnResultChange: values["fail-on-result-change"],
199
216
  allowRemoved: values["allow-removed"],
217
+ allowChangedInputs: values["allow-changed-inputs"],
200
218
  };
201
219
  const diff = diffReports(await readReport(positionals[0]), await readReport(positionals[1]), options);
202
220
  console.log(values.markdown
@@ -399,7 +417,8 @@ async function main() {
399
417
  console.log(table(p));
400
418
  }
401
419
  const mismatch = p.nodeCycles !== undefined && p.nodeCycles !== p.measuredCycles;
402
- return mismatch || p.groups.some((g) => g.result !== 0) ? 1 : 0;
420
+ // Over the node's per-transaction limit: the node would reject it, so a verdict too.
421
+ return mismatch || p.groups.some((g) => g.result !== 0) || p.overTxLimit !== undefined ? 1 : 0;
403
422
  }
404
423
  // 1 is a verdict (a script rejects, a sum differs, a diff over its thresholds);
405
424
  // anything thrown is the tool not getting that far: 2, so a CI job asserting a
package/dist/diff.d.ts CHANGED
@@ -17,6 +17,61 @@ export interface CycleReport {
17
17
  * fails on these, from either side.
18
18
  */
19
19
  errors?: string[];
20
+ /**
21
+ * What the numbers were measured with (1.3.0 and later). A diff checks the two
22
+ * sides' manifests: changed scenario inputs, another debugger, another CKB-VM.
23
+ */
24
+ manifest?: ReportManifest;
25
+ }
26
+ /**
27
+ * A reproducibility manifest. Every string in it can come from a pull request
28
+ * (a path in its cycles.json, the toolchain its build printed): whatever shows
29
+ * it cleans it as any other report text.
30
+ */
31
+ export interface ReportManifest {
32
+ /** This object's layout. */
33
+ schema: 1;
34
+ /** The package that wrote the report, e.g. `{ name: "ckb-cycle-tools", version: "1.3.0" }`. */
35
+ tool: {
36
+ name: string;
37
+ version: string;
38
+ };
39
+ debugger: {
40
+ /** `--version`, as the report's `debugger`. */
41
+ version: string;
42
+ /** The native binary every scenario ran, and its sha256. */
43
+ path?: string;
44
+ sha256?: string;
45
+ /** For a WebAssembly build: its id (the sha256 of the .wasm). */
46
+ wasm?: string;
47
+ };
48
+ /** The commit checked out where the cycles.json is, and where the binaries came from with --binaries-from. */
49
+ git?: {
50
+ commit?: string;
51
+ binaries?: string;
52
+ };
53
+ /** Free text the caller gave (`--toolchain`): the compiler and build profile, e.g. `rustc -V`. */
54
+ toolchain?: string;
55
+ scenarios: ScenarioManifest[];
56
+ }
57
+ export interface ScenarioManifest {
58
+ name: string;
59
+ /** The CKB-VM version the scenario's `hash_type: "type"` scripts ran on. */
60
+ scriptVersion?: number;
61
+ /**
62
+ * sha256 over what decides what the scenario runs: its cycles.json entry
63
+ * (canonical, without its name or file path) and its mock or transaction file
64
+ * as read. Two reports with the same scenario name and a different hash did
65
+ * not measure the same thing.
66
+ */
67
+ inputs?: string;
68
+ /** The binaries swapped in, as read: the path the config gives, sha256, size in bytes. */
69
+ binaries?: {
70
+ name: string;
71
+ path: string;
72
+ sha256: string;
73
+ size: number;
74
+ }[];
20
75
  }
21
76
  export interface ScenarioResult {
22
77
  name: string;
@@ -38,7 +93,35 @@ export interface ScenarioResult {
38
93
  */
39
94
  binaryMissing?: boolean;
40
95
  missingBinaries?: string[];
96
+ /**
97
+ * Set when some group rejects (added after 1.2.0): the selector of the group the
98
+ * node would stop at, the first failing one in its order (lock groups, then
99
+ * type groups, each by script hash). The node reports no cycles for a
100
+ * rejected transaction, so the groups' cycles, each run on its own, do not
101
+ * add up to anything the node spends.
102
+ */
103
+ nodeStopsAt?: string;
104
+ /**
105
+ * Set only when the groups' cycles add up to more than the node's limit for
106
+ * one transaction (added after 1.3.0): `limit` is the consensus
107
+ * `max_block_cycles` (3,500,000,000 on mainnet and testnet), `sum` the groups'
108
+ * cycles. ckb-debugger gives each group that limit on its own; the node gives
109
+ * it to the whole transaction, so it would reject this one (exceeded max
110
+ * cycles). `nodeStopsAt` names the group where the running sum passes it.
111
+ */
112
+ overTxLimit?: {
113
+ limit: number;
114
+ sum: number;
115
+ };
41
116
  }
117
+ /**
118
+ * A scenario's `overTxLimit`, read defensively (a report file is the PR's):
119
+ * undefined unless both numbers are there and the sum is over the limit.
120
+ */
121
+ export declare function overTxLimitOf(s: ScenarioResult): {
122
+ limit: number;
123
+ sum: number;
124
+ } | undefined;
42
125
  export interface GroupResult {
43
126
  /** ckb-debugger's selector, e.g. `input.0.lock`: the key a group is matched by across reports. */
44
127
  selector: string;
@@ -52,6 +135,14 @@ export interface GroupResult {
52
135
  error?: string;
53
136
  /** Cycles; 0 when the VM stopped the script (`error`), whose count is not a cost. */
54
137
  cycles: number;
138
+ /**
139
+ * Set on every group of a scenario where some group rejects (added after 1.2.0):
140
+ * whether the node would run this group at all. It stops at the first failing
141
+ * group in its order (the scenario's `nodeStopsAt`); every group is still run
142
+ * on its own here, so the ones after it have a result and cycles the node
143
+ * would never produce. Absent when every group accepts: the node runs them all.
144
+ */
145
+ nodeReaches?: boolean;
55
146
  /**
56
147
  * The binary this group's code stands for was not in the checkout (the base):
57
148
  * its numbers are the mock's code, not that build's, so it is not compared.
@@ -125,8 +216,20 @@ export interface DiffRow {
125
216
  spawned?: number;
126
217
  /** With `spawned`: the cycles the head's function profile covers. */
127
218
  profiled?: number;
128
- /** Why the base side of this group was not compared (its binary was not in the base checkout). */
219
+ /** Why the base side of this group was not compared (its binary was not in the base checkout, its inputs changed). */
129
220
  notCompared?: string;
221
+ /** Not compared because the scenario's inputs differ between the two reports' manifests. */
222
+ inputsChanged?: boolean;
223
+ }
224
+ /** One side's manifest, as a comment shows it. */
225
+ export interface ManifestSummary {
226
+ tool?: string;
227
+ debugger?: string;
228
+ debuggerSha256?: string;
229
+ wasm?: string;
230
+ commit?: string;
231
+ binariesCommit?: string;
232
+ toolchain?: string;
130
233
  }
131
234
  export interface CycleDiff {
132
235
  rows: DiffRow[];
@@ -161,6 +264,40 @@ export interface CycleDiff {
161
264
  head: string;
162
265
  };
163
266
  threshold?: number;
267
+ /**
268
+ * Groups not compared because their scenario's inputs (mock, transaction
269
+ * file, cycles.json entry) differ between the manifests. A gated diff fails
270
+ * on them unless `allowChangedInputs`.
271
+ */
272
+ inputsChanged?: number;
273
+ /** The scenarios whose inputs changed. */
274
+ changedScenarios?: string[];
275
+ /**
276
+ * Why the two reports did not run in the same environment, from their
277
+ * manifests: another ckb-debugger (version or binary), another CKB-VM version
278
+ * for a scenario. A gated diff fails on any.
279
+ */
280
+ environment?: string[];
281
+ /** What could not be checked, or differs without failing: a report without a manifest, another toolchain. */
282
+ notes?: string[];
283
+ /** Each side's manifest, short, for the comment. */
284
+ manifests?: {
285
+ base?: ManifestSummary;
286
+ head?: ManifestSummary;
287
+ };
288
+ /**
289
+ * Head scenarios whose groups add up to more than the node's per-transaction
290
+ * limit while the base's did not (`newScenario`: no base scenario ran to
291
+ * compare with). The node would reject them (exceeded max cycles) though every
292
+ * group may accept: a gated diff fails on one the base ran under the limit, as
293
+ * on a group that starts rejecting; `failOnResultChange` on any.
294
+ */
295
+ overTxLimit?: {
296
+ scenario: string;
297
+ limit: number;
298
+ sum: number;
299
+ newScenario?: boolean;
300
+ }[];
164
301
  }
165
302
  export interface DiffOptions {
166
303
  /** Percent. An increase above it is flagged, and makes `failed` true. */
@@ -169,7 +306,11 @@ export interface DiffOptions {
169
306
  failOnResultChange?: boolean;
170
307
  /** Removed, renamed or uncompared base groups do not fail a gated diff. */
171
308
  allowRemoved?: boolean;
309
+ /** Scenarios whose inputs changed (manifest hashes differ) do not fail a gated diff; still not compared. */
310
+ allowChangedInputs?: boolean;
172
311
  }
312
+ /** The note on a group whose scenario's inputs differ between base and head. */
313
+ export declare const INPUTS_CHANGED = "inputs changed: the scenario's mock, transaction file or cycles.json entry differs from the base's";
173
314
  /** The message for a binary no scenario's cell deps match, in a report's `errors` and the diff. */
174
315
  export declare const unmatchedError: (name: string) => string;
175
316
  /**
@@ -192,7 +333,16 @@ export declare function functionDeltas(base: FunctionResult[] | undefined, head:
192
333
  * run, an error in either report, and a base group with nothing to compare
193
334
  * against (removed, renamed, moved, or measured without its binary) unless
194
335
  * `allowRemoved`. A head group the VM stopped is a result change, and over any
195
- * `failAbove` threshold.
336
+ * `failAbove` threshold. With a gate set, a group that accepted on the base and
337
+ * rejects on the head (`ok` → `exit N` or a VM error) always fails, whatever its
338
+ * cycles: a cheaper rejection is not a saving. `failOnResultChange` fails on any
339
+ * result change besides (reject → accept, one exit code or VM error to another,
340
+ * a new group that does not accept). A new group has no base that accepted, so
341
+ * this rule leaves it alone: with `failAbove` alone it fails only as before (a
342
+ * new group the VM stopped is over the threshold). A scenario whose groups add
343
+ * up to more than the node's per-transaction limit on the head, and not on the
344
+ * base, counts as a group that starts rejecting (the node would reject it); a
345
+ * new scenario over the limit, as a new group that does not accept.
196
346
  */
197
347
  export declare function diffFailed(diff: CycleDiff, options?: DiffOptions): boolean;
198
348
  /**
@@ -234,5 +384,10 @@ export interface MarkdownOptions {
234
384
  * reverse) matters more than any cycle count.
235
385
  */
236
386
  export declare function formatMarkdown(diff: CycleDiff, options?: MarkdownOptions): string;
387
+ /**
388
+ * One line per side that has a manifest: tool, debugger (and its binary's
389
+ * hash), commit, toolchain. Plain text: the caller cleans it (md or termLine).
390
+ */
391
+ export declare function manifestLines(diff: CycleDiff): string[];
237
392
  /** The same table for a terminal. */
238
393
  export declare function formatText(diff: CycleDiff): string;