vysort 0.1.0__tar.gz

vysort-0.1.0/PKG-INFO

@@ -0,0 +1,127 @@
+Metadata-Version: 2.3
+Name: vysort
+Version: 0.1.0
+Summary: Make vyper 0.3.4-0.3.7 bytecode deterministic by forcing the internal-function layout
+Author: banteg
+Author-email: banteg <4562643+banteg@users.noreply.github.com>
+Requires-Dist: uv
+Requires-Python: >=3.10
+Description-Content-Type: text/markdown
+
+# vysort
+
+Make vyper 0.3.4–0.3.7 bytecode deterministic.
+
+These compiler versions emit nondeterministic bytecode for any contract whose
+call graph contains a *decision point* — a function calling ≥2 internal
+functions defined later in the file ([vyper#3369](https://github.com/vyperlang/vyper/issues/3369)).
+The internal-function sections get permuted per environment (and per run on
+linux), which blocks byte-exact verification: the verifier's recompile may
+never reproduce what the deployer's machine happened to emit.
+
+vysort fixes this with no compiler modifications: it decodes the deployed
+layout straight off the on-chain bytecode, then reorders the source so
+internal function defs come first, in that exact order. The topsort then has
+zero decision points and a stock compiler produces the deployed bytecode
+everywhere, every run.
+
+## Install
+
+```sh
+uv tool install vysort
+```
+
+Or run from a checkout: `uv run vysort ...`
+
+vysort itself runs on any modern python and depends only on uv. The
+vyper-touching work runs in an ephemeral `uv run` environment with the
+matching compiler: the vyper version is auto-detected from the source's
+version pragma (override with `--vyper`), on python 3.10 by default
+(override with `--python`). No old python or vyper install needed.
+
+## Verify your contract
+
+If your vyper 0.3.x contract fails verification, this is the command:
+
+```sh
+vysort verify contract.vy --address 0x2cced4ff... --rpc-url https://eth.drpc.org
+```
+
+It fetches the deployed code and chain id from the RPC, recovers the deployed
+internal-function layout from the on-chain bytes, rewrites the source to force
+that layout, confirms the exact standard-json payload reproduces the runtime
+byte-for-byte (a preflight compile through vyper's own std-json entry point —
+the same path the verifier's binary takes), and submits it to sourcify's v2
+API with a stock compiler version. No forks, no patched binaries, no special
+verifier support.
+
+Use `--dry-run` to inspect the submission payload without sending it,
+`--creation-tx` to help the creation match, `--sourcify-url` to target
+another server, and `-o` to keep the rewritten source.
+
+Note: creation matches are only guaranteed when `__init__` calls ≤1 internal
+function; the init-callee section of creation code is not forced by source
+order. Runtime matches are always forceable.
+
+## Match without submitting
+
+To recover the layout and prove the match locally — against on-chain code or
+a hex file — without involving a verifier:
+
+```sh
+vysort match contract.vy --address 0x2cced4ff... --rpc-url https://eth.drpc.org -o matched.vy
+vysort match contract.vy --runtime runtime.hex -o matched.vy
+```
+
+The deployed layout is recovered in 2 compiles regardless of contract size:
+one instrumented compile maps each internal function's section boundaries and
+masks the layout-dependent address bytes, the deployed order is then decoded
+straight off the on-chain bytes, and one reordered stock compile verifies it
+byte-exactly — `exact`, or `prefix` when the deployed code carries an appended
+immutable tail. If the decode hits an edge case, reachable layouts are
+brute-forced one compile at a time as a fallback. For unaffected compiler
+versions a single compile-and-compare runs instead. `--evm-version istanbul`
+helps pre-berlin deployments whose nonreentrant lock constants differ.
+
+The matched source written by `-o` is ordinary vyper that any stock compiler
+of that version turns into the deployed bytecode — auxdata contains no source
+hash, so the output is byte-identical to what the original source produces
+under that ordering.
+
+## Developer curiosities
+
+The remaining subcommands expose the machinery.
+
+Analyze a contract for ordering nondeterminism:
+
+```sh
+vysort check contract.vy
+```
+
+```json
+{
+  "internal_fns": 2,
+  "decision_points": 1,
+  "reachable_layouts": 2,
+  "immune": false,
+  "env_layout": ["_triple", "_double"],
+  "layouts": [["_double", "_triple"], ["_triple", "_double"]]
+}
+```
+
+`immune: true` means exactly one layout is reachable — the contract was never
+at risk; this covers 93% of affected-band mainnet contracts. Otherwise
+`layouts` (when small) enumerates every layout the deployer's heap could have
+produced. The check is version-aware: sources targeting compilers outside the
+affected 0.3.4–0.3.7 band short-circuit to `immune: true` without compiling.
+
+Force an arbitrary layout by rewriting the source:
+
+```sh
+vysort reorder contract.vy _double,_triple -o forced.vy
+vysort reorder contract.vy layout.json > forced.vy
+```
+
+The layout is a comma-separated list of internal function names or a JSON
+file (`["_double", "_triple"]`). This is the forcing primitive `match` and
+`verify` are built on.

vysort-0.1.0/README.md

@@ -0,0 +1,117 @@
+# vysort
+
+Make vyper 0.3.4–0.3.7 bytecode deterministic.
+
+These compiler versions emit nondeterministic bytecode for any contract whose
+call graph contains a *decision point* — a function calling ≥2 internal
+functions defined later in the file ([vyper#3369](https://github.com/vyperlang/vyper/issues/3369)).
+The internal-function sections get permuted per environment (and per run on
+linux), which blocks byte-exact verification: the verifier's recompile may
+never reproduce what the deployer's machine happened to emit.
+
+vysort fixes this with no compiler modifications: it decodes the deployed
+layout straight off the on-chain bytecode, then reorders the source so
+internal function defs come first, in that exact order. The topsort then has
+zero decision points and a stock compiler produces the deployed bytecode
+everywhere, every run.
+
+## Install
+
+```sh
+uv tool install vysort
+```
+
+Or run from a checkout: `uv run vysort ...`
+
+vysort itself runs on any modern python and depends only on uv. The
+vyper-touching work runs in an ephemeral `uv run` environment with the
+matching compiler: the vyper version is auto-detected from the source's
+version pragma (override with `--vyper`), on python 3.10 by default
+(override with `--python`). No old python or vyper install needed.
+
+## Verify your contract
+
+If your vyper 0.3.x contract fails verification, this is the command:
+
+```sh
+vysort verify contract.vy --address 0x2cced4ff... --rpc-url https://eth.drpc.org
+```
+
+It fetches the deployed code and chain id from the RPC, recovers the deployed
+internal-function layout from the on-chain bytes, rewrites the source to force
+that layout, confirms the exact standard-json payload reproduces the runtime
+byte-for-byte (a preflight compile through vyper's own std-json entry point —
+the same path the verifier's binary takes), and submits it to sourcify's v2
+API with a stock compiler version. No forks, no patched binaries, no special
+verifier support.
+
+Use `--dry-run` to inspect the submission payload without sending it,
+`--creation-tx` to help the creation match, `--sourcify-url` to target
+another server, and `-o` to keep the rewritten source.
+
+Note: creation matches are only guaranteed when `__init__` calls ≤1 internal
+function; the init-callee section of creation code is not forced by source
+order. Runtime matches are always forceable.
+
+## Match without submitting
+
+To recover the layout and prove the match locally — against on-chain code or
+a hex file — without involving a verifier:
+
+```sh
+vysort match contract.vy --address 0x2cced4ff... --rpc-url https://eth.drpc.org -o matched.vy
+vysort match contract.vy --runtime runtime.hex -o matched.vy
+```
+
+The deployed layout is recovered in 2 compiles regardless of contract size:
+one instrumented compile maps each internal function's section boundaries and
+masks the layout-dependent address bytes, the deployed order is then decoded
+straight off the on-chain bytes, and one reordered stock compile verifies it
+byte-exactly — `exact`, or `prefix` when the deployed code carries an appended
+immutable tail. If the decode hits an edge case, reachable layouts are
+brute-forced one compile at a time as a fallback. For unaffected compiler
+versions a single compile-and-compare runs instead. `--evm-version istanbul`
+helps pre-berlin deployments whose nonreentrant lock constants differ.
+
+The matched source written by `-o` is ordinary vyper that any stock compiler
+of that version turns into the deployed bytecode — auxdata contains no source
+hash, so the output is byte-identical to what the original source produces
+under that ordering.
+
+## Developer curiosities
+
+The remaining subcommands expose the machinery.
+
+Analyze a contract for ordering nondeterminism:
+
+```sh
+vysort check contract.vy
+```
+
+```json
+{
+  "internal_fns": 2,
+  "decision_points": 1,
+  "reachable_layouts": 2,
+  "immune": false,
+  "env_layout": ["_triple", "_double"],
+  "layouts": [["_double", "_triple"], ["_triple", "_double"]]
+}
+```
+
+`immune: true` means exactly one layout is reachable — the contract was never
+at risk; this covers 93% of affected-band mainnet contracts. Otherwise
+`layouts` (when small) enumerates every layout the deployer's heap could have
+produced. The check is version-aware: sources targeting compilers outside the
+affected 0.3.4–0.3.7 band short-circuit to `immune: true` without compiling.
+
+Force an arbitrary layout by rewriting the source:
+
+```sh
+vysort reorder contract.vy _double,_triple -o forced.vy
+vysort reorder contract.vy layout.json > forced.vy
+```
+
+The layout is a comma-separated list of internal function names or a JSON
+file (`["_double", "_triple"]`). This is the forcing primitive `match` and
+`verify` are built on.

vysort-0.1.0/pyproject.toml

@@ -0,0 +1,17 @@
+[project]
+name = "vysort"
+version = "0.1.0"
+description = "Make vyper 0.3.4-0.3.7 bytecode deterministic by forcing the internal-function layout"
+readme = "README.md"
+authors = [
+    { name = "banteg", email = "4562643+banteg@users.noreply.github.com" }
+]
+requires-python = ">=3.10"
+dependencies = ["uv"]
+
+[project.scripts]
+vysort = "vysort:main"
+
+[build-system]
+requires = ["uv_build>=0.11.17,<0.12.0"]
+build-backend = "uv_build"

vysort-0.1.0/src/vysort/__init__.py

@@ -0,0 +1,343 @@
+"""vysort: make vyper 0.3.4-0.3.7 bytecode deterministic by forcing the
+internal-function layout through source definition reordering.
+
+The CLI runs on any modern python; vyper-touching work is delegated to
+self-contained worker scripts executed via `uv run` in an ephemeral
+environment with the matching python and vyper versions.
+"""
+
+import argparse
+import json
+import pathlib
+import re
+import subprocess
+import sys
+import tempfile
+import time
+import urllib.error
+import urllib.request
+
+from uv import find_uv_bin
+
+WORKERS = pathlib.Path(__file__).parent / "workers"
+
+# the whole affected band (vyper 0.3.4-0.3.7) supports python 3.10
+DEFAULT_PYTHON = "3.10"
+
+# call-graph ordering nondeterminism: introduced in 0.3.4, fixed in 0.3.8
+AFFECTED_MIN, AFFECTED_MAX = (0, 3, 4), (0, 3, 7)
+
+DEFAULT_SOURCIFY = "https://sourcify.dev/server"
+
+
+def version_tuple(version: str) -> tuple:
+    m = re.match(r"(\d+)\.(\d+)\.(\d+)", version)
+    if not m:
+        raise SystemExit(f"cannot parse vyper version: {version!r}")
+    return tuple(int(x) for x in m.groups())
+
+
+def is_affected(version: str) -> bool:
+    return AFFECTED_MIN <= version_tuple(version) <= AFFECTED_MAX
+
+
+def detect_vyper_version(src: str) -> str:
+    m = re.search(r"^#\s*(?:@version|pragma\s+version)\s+(.+)$", src, re.M)
+    if not m:
+        raise SystemExit("no version pragma in source; pass --vyper")
+    spec = m.group(1).strip()
+    exact = re.fullmatch(r"==?\s*(\d+\.\d+\.\d+\S*)|(\d+\.\d+\.\d+\S*)", spec)
+    if not exact:
+        raise SystemExit(
+            f"version pragma {spec!r} is a range, not a concrete compiler; "
+            "pass --vyper with the version the contract was deployed with"
+        )
+    return exact.group(1) or exact.group(2)
+
+
+def run_worker(worker: str, args, vyper_version: str, python: str) -> str:
+    cmd = [
+        find_uv_bin(),
+        "run",
+        "--quiet",
+        "--no-project",
+        "--python", python,
+        "--with", f"vyper=={vyper_version}",
+        str(WORKERS / f"{worker}.py"),
+        *map(str, args),
+    ]
+    r = subprocess.run(cmd, stdout=subprocess.PIPE, text=True)
+    if r.returncode:
+        sys.exit(r.returncode)
+    return r.stdout
+
+
+def http_json(url: str, payload: dict | None = None, method: str = "POST") -> dict:
+    req = urllib.request.Request(
+        url,
+        data=json.dumps(payload).encode() if payload is not None else None,
+        method=method,
+        headers={"Content-Type": "application/json", "User-Agent": "vysort"},
+    )
+    try:
+        with urllib.request.urlopen(req, timeout=30) as r:
+            return json.load(r)
+    except urllib.error.HTTPError as e:
+        raise SystemExit(f"http {e.code} from {url}: {e.read().decode(errors='replace')}")
+    except (urllib.error.URLError, TimeoutError) as e:
+        raise SystemExit(f"cannot reach {url}: {e}")
+
+
+def eth_rpc(rpc_url: str, method: str, params: list):
+    reply = http_json(rpc_url, {"jsonrpc": "2.0", "method": method, "params": params, "id": 1})
+    if "error" in reply:
+        raise SystemExit(f"rpc error from {method}: {reply['error']}")
+    return reply["result"]
+
+
+def fetch_runtime(address: str, rpc_url: str) -> str:
+    code = eth_rpc(rpc_url, "eth_getCode", [address, "latest"])
+    if code in (None, "0x"):
+        raise SystemExit(f"no code at {address}")
+    return code
+
+
+def resolve_runtime(args, parser) -> str:
+    if args.runtime:
+        return args.runtime.read_text()
+    if args.address and args.rpc_url:
+        return fetch_runtime(args.address, args.rpc_url)
+    parser.error("supply --runtime or --address with --rpc-url")
+
+
+def run_match(
+    source: pathlib.Path, runtime_hex: str, vyper_version: str, python: str, evm_version: str | None
+) -> dict:
+    mode = "brute" if is_affected(vyper_version) else "single"
+    clean = "".join(runtime_hex.split()).removeprefix("0x")
+    with tempfile.NamedTemporaryFile("w", suffix=".hex") as f:
+        f.write(clean)
+        f.flush()
+        worker_args = [source, f.name, mode] + ([evm_version] if evm_version else [])
+        out = run_worker("match", worker_args, vyper_version, python)
+    return json.loads(out)
+
+
+def run_preflight(payload: dict, runtime_hex: str, vyper_version: str, python: str) -> dict:
+    """Compile the exact submission payload via vyper's std-json path and
+    compare against the deployed runtime."""
+    clean = "".join(runtime_hex.split()).removeprefix("0x")
+    with tempfile.NamedTemporaryFile("w", suffix=".json") as fj, tempfile.NamedTemporaryFile(
+        "w", suffix=".hex"
+    ) as fh:
+        json.dump(payload["stdJsonInput"], fj)
+        fj.flush()
+        fh.write(clean)
+        fh.flush()
+        out = run_worker(
+            "preflight", [fj.name, fh.name, payload["contractIdentifier"]], vyper_version, python
+        )
+    return json.loads(out)
+
+
+def compiler_version_string(vyper_version: str, python: str) -> str:
+    """Full version with commit hash as sourcify expects it, e.g. 0.3.7+commit.6020b8bb.
+
+    The canonical string comes from the official release binary names (pip vyper
+    reports a shorter git hash than the release assets carry)."""
+    try:
+        release = http_json(
+            f"https://api.github.com/repos/vyperlang/vyper/releases/tags/v{vyper_version}",
+            method="GET",
+        )
+        for asset in release["assets"]:
+            m = re.match(rf"vyper\.({re.escape(vyper_version)}\+commit\.[0-9a-f]+)\.", asset["name"])
+            if m:
+                return m.group(1)
+    except SystemExit:
+        pass
+    cmd = [
+        find_uv_bin(),
+        "run",
+        "--quiet",
+        "--no-project",
+        "--python", python,
+        "--with", f"vyper=={vyper_version}",
+        "vyper", "--version",
+    ]
+    return subprocess.run(cmd, stdout=subprocess.PIPE, text=True, check=True).stdout.strip()
+
+
+def build_std_json(filename: str, source: str, evm_version: str | None) -> dict:
+    settings = {"outputSelection": {"*": ["evm.bytecode", "evm.deployedBytecode", "abi"]}}
+    if evm_version:
+        settings["evmVersion"] = evm_version
+    return {"language": "Vyper", "sources": {filename: {"content": source}}, "settings": settings}
+
+
+def sourcify_verify(sourcify_url: str, chain_id: int, address: str, payload: dict) -> dict:
+    submit = http_json(f"{sourcify_url}/v2/verify/{chain_id}/{address}", payload)
+    vid = submit["verificationId"]
+    print(f"submitted verification job {vid}", file=sys.stderr)
+    deadline = time.monotonic() + 600
+    while time.monotonic() < deadline:
+        job = http_json(f"{sourcify_url}/v2/verify/{vid}", method="GET")
+        if job.get("isJobCompleted"):
+            return job
+        time.sleep(2)
+    raise SystemExit(f"verification job {vid} did not complete within 10 minutes")
+
+
+def parse_layout(value: str) -> list:
+    """Layout is a JSON file path or an inline comma-separated list of names."""
+    path = pathlib.Path(value)
+    if path.exists():
+        return json.loads(path.read_text())
+    return [n.strip() for n in value.split(",") if n.strip()]
+
+
+def main() -> None:
+    parser = argparse.ArgumentParser(prog="vysort", description=__doc__)
+    common = argparse.ArgumentParser(add_help=False)
+    common.add_argument("source", type=pathlib.Path, help="vyper source file")
+    common.add_argument("--vyper", help="vyper version (default: from source pragma)")
+    common.add_argument(
+        "--python", default=DEFAULT_PYTHON, help=f"python for the compiler env (default: {DEFAULT_PYTHON})"
+    )
+    onchain = argparse.ArgumentParser(add_help=False)
+    onchain.add_argument("--runtime", type=pathlib.Path, help="file with the target runtime bytecode hex")
+    onchain.add_argument("--address", help="contract address to fetch the runtime code from")
+    onchain.add_argument("--rpc-url", help="json-rpc endpoint for --address")
+
+    sub = parser.add_subparsers(dest="command", required=True)
+
+    sub.add_parser(
+        "check",
+        parents=[common],
+        help="analyze a contract for ordering nondeterminism (decision points, reachable layouts)",
+    )
+
+    p_reorder = sub.add_parser(
+        "reorder",
+        parents=[common],
+        help="rewrite source so a stock compiler deterministically produces the target layout",
+    )
+    p_reorder.add_argument(
+        "layout",
+        type=parse_layout,
+        help='target internal layout: JSON file (["_f1", ...]) or comma-separated names (_f1,_f2)',
+    )
+    p_reorder.add_argument(
+        "-o", "--output", type=pathlib.Path, help="write reordered source here (default: stdout)"
+    )
+
+    p_match = sub.add_parser(
+        "match",
+        parents=[common, onchain],
+        help="find the deployed layout: compile reachable layouts until the runtime matches",
+    )
+    p_match.add_argument("--evm-version", help="evm version for compilation (e.g. istanbul)")
+    p_match.add_argument(
+        "-o", "--output", type=pathlib.Path, help="write the matched reordered source here"
+    )
+
+    p_verify = sub.add_parser(
+        "verify",
+        parents=[common],
+        help="match the deployed layout and submit the source to sourcify",
+    )
+    p_verify.add_argument("--address", required=True, help="deployed contract address")
+    p_verify.add_argument("--rpc-url", required=True, help="json-rpc endpoint (also provides the chain id)")
+    p_verify.add_argument("--sourcify-url", default=DEFAULT_SOURCIFY, help=f"sourcify server (default: {DEFAULT_SOURCIFY})")
+    p_verify.add_argument("--creation-tx", help="contract creation tx hash (helps the creation match)")
+    p_verify.add_argument("--evm-version", help="evm version for compilation (e.g. istanbul)")
+    p_verify.add_argument(
+        "--dry-run", action="store_true", help="print the submission payload instead of submitting"
+    )
+    p_verify.add_argument(
+        "-o", "--output", type=pathlib.Path, help="write the matched reordered source here"
+    )
+
+    args = parser.parse_args()
+    vyper_version = args.vyper or detect_vyper_version(args.source.read_text())
+
+    if args.command == "check":
+        if not is_affected(vyper_version):
+            report = {
+                "vyper": vyper_version,
+                "affected": False,
+                "immune": True,
+                "reason": "call-graph ordering bug only affects vyper 0.3.4-0.3.7",
+            }
+        else:
+            report = {"vyper": vyper_version, "affected": True}
+            report.update(json.loads(run_worker("check", [args.source], vyper_version, args.python)))
+        print(json.dumps(report, indent=2))
+
+    elif args.command == "reorder":
+        out = run_worker("reorder", [args.source, json.dumps(args.layout)], vyper_version, args.python)
+        if args.output:
+            args.output.write_text(out)
+        else:
+            sys.stdout.write(out)
+
+    elif args.command == "match":
+        runtime_hex = resolve_runtime(args, parser)
+        result = run_match(args.source, runtime_hex, vyper_version, args.python, args.evm_version)
+        source = result.pop("source", None)
+        if args.output and source:
+            args.output.write_text(source)
+            result["output"] = str(args.output)
+        print(json.dumps({"vyper": vyper_version, "affected": is_affected(vyper_version), **result}, indent=2))
+        if result["status"] not in ("exact", "prefix"):
+            sys.exit(1)
+
+    elif args.command == "verify":
+        runtime_hex = fetch_runtime(args.address, args.rpc_url)
+        chain_id = int(eth_rpc(args.rpc_url, "eth_chainId", []), 16)
+        result = run_match(args.source, runtime_hex, vyper_version, args.python, args.evm_version)
+        source = result.pop("source", None)
+        if result["status"] not in ("exact", "prefix"):
+            print(json.dumps(result, indent=2))
+            raise SystemExit(f"runtime matches no reachable layout (status: {result['status']}); not submitting")
+        if args.output:
+            args.output.write_text(source)
+        filename = args.source.name
+        payload = {
+            "stdJsonInput": build_std_json(filename, source, args.evm_version),
+            "compilerVersion": compiler_version_string(vyper_version, args.python),
+            "contractIdentifier": f"{filename}:{args.source.stem}",
+        }
+        if args.creation_tx:
+            payload["creationTransactionHash"] = args.creation_tx
+        preflight = run_preflight(payload, runtime_hex, vyper_version, args.python)
+        if preflight["status"] not in ("exact", "prefix"):
+            print(json.dumps({"match": result, "preflight": preflight}, indent=2))
+            raise SystemExit(
+                f"preflight std-json compile does not reproduce the runtime "
+                f"(status: {preflight['status']}); not submitting"
+            )
+        print(f"preflight: std-json payload reproduces the runtime ({preflight['status']})", file=sys.stderr)
+        if args.dry_run:
+            print(json.dumps(
+                {
+                    "chain_id": chain_id,
+                    "address": args.address,
+                    "match": result,
+                    "preflight": preflight,
+                    "payload": payload,
+                },
+                indent=2,
+            ))
+            return
+        job = sourcify_verify(args.sourcify_url, chain_id, args.address, payload)
+        report = {
+            "match": result["status"],
+            "preflight": preflight["status"],
+            "layout": result.get("layout"),
+            "contract": job.get("contract"),
+            "error": job.get("error"),
+        }
+        print(json.dumps(report, indent=2))
+        if not (job.get("contract") or {}).get("match"):
+            sys.exit(1)

vysort-0.1.0/src/vysort/workers/check.py

@@ -0,0 +1,142 @@
+"""Decision-point analysis for vyper 0.3.4-0.3.7 ordering nondeterminism.
+
+The bug requires multiple valid topsorts — i.e. some called_functions set
+holding >=2 not-yet-placed internal functions at DFS time. Contracts without
+such a decision point have exactly one reachable layout and are immune.
+
+Captures the call graph + source order from a real compile, enumerates (or
+samples) all per-set iteration orders, and counts distinct reachable internal
+layouts.
+
+Self-contained worker: runs in an ephemeral `uv run --with vyper==X` env and
+imports nothing from vysort. argv: <source.vy>; JSON report on stdout.
+"""
+
+import itertools
+import json
+import pathlib
+import random
+import sys
+
+import vyper.codegen.module as module_mod
+from vyper.compiler.phases import CompilerData
+
+ENUMERATION_CAP = 20000
+
+
+def capture_call_graph(src: str) -> dict:
+    """Compile once with an instrumented _topsort; return the call graph."""
+    captured = {}
+    orig_topsort = module_mod._topsort
+
+    def capturing_topsort(functions):
+        if not captured:
+            for f in functions:
+                t = f._metadata["type"]
+                captured[f.name] = {
+                    "callees": [c.name for c in t.called_functions],  # env iteration order
+                    "internal": t.is_internal,
+                }
+            captured["__source_order__"] = [f.name for f in functions]
+        return orig_topsort(functions)
+
+    module_mod._topsort = capturing_topsort
+    try:
+        CompilerData(src).bytecode_runtime  # triggers codegen -> capture
+    finally:
+        module_mod._topsort = orig_topsort
+    return captured
+
+
+def simulate(source_order, callee_order, internal):
+    """Replicate _topsort + dedup-first; return internal layout tuple."""
+    placed = []
+    placed_set = set()
+
+    def helper(f):
+        if f in placed_set:
+            return
+        for c in callee_order[f]:
+            helper(c)
+        if f not in placed_set:
+            placed.append(f)
+            placed_set.add(f)
+
+    for f in source_order:
+        helper(f)
+    return tuple(f for f in placed if internal.get(f))
+
+
+def decision_points(source_order, callee_order, internal):
+    """Count sets that, at first visit, contain >=2 unplaced callees."""
+    points = []
+    placed = set()
+
+    def helper(f):
+        if f in placed:
+            return
+        unplaced = [c for c in callee_order[f] if c not in placed]
+        if len([c for c in unplaced if internal.get(c)]) >= 2:
+            points.append((f, len(unplaced)))
+        for c in callee_order[f]:
+            helper(c)
+        placed.add(f)
+
+    for f in source_order:
+        helper(f)
+    return points
+
+
+def enumerate_layouts(source_order, callee_order, internal):
+    """Enumerate per-set permutations (cartesian product), cap with sampling.
+
+    Returns (layouts, exact, choice_space)."""
+    multi = [n for n, c in callee_order.items() if len(c) >= 2]
+    space = 1
+    for n in multi:
+        for i in range(2, len(callee_order[n]) + 1):
+            space *= i
+    layouts = set()
+    if space <= ENUMERATION_CAP:
+        pools = [list(itertools.permutations(callee_order[n])) for n in multi]
+        for combo in itertools.product(*pools):
+            co = dict(callee_order)
+            co.update({n: list(p) for n, p in zip(multi, combo)})
+            layouts.add(simulate(source_order, co, internal))
+        return layouts, True, space
+    rng = random.Random(1)
+    for _ in range(ENUMERATION_CAP):
+        co = dict(callee_order)
+        for n in multi:
+            p = callee_order[n][:]
+            rng.shuffle(p)
+            co[n] = p
+        layouts.add(simulate(source_order, co, internal))
+    return layouts, False, space
+
+
+def check(src: str) -> dict:
+    captured = capture_call_graph(src)
+    source_order = captured.pop("__source_order__")
+    internal = {n: v["internal"] for n, v in captured.items()}
+    base_callees = {n: v["callees"] for n, v in captured.items()}
+
+    env_layout = simulate(source_order, base_callees, internal)
+    dps = decision_points(source_order, base_callees, internal)
+    layouts, exact, space = enumerate_layouts(source_order, base_callees, internal)
+
+    return {
+        "internal_fns": sum(internal.values()),
+        "decision_points": len(dps),
+        "choice_space": space,
+        "reachable_layouts": len(layouts),
+        "reachable_exact": exact,
+        "immune": len(layouts) == 1 and exact,
+        "env_layout": list(env_layout),
+        "layouts": sorted(layouts) if exact and len(layouts) <= 64 else None,
+    }
+
+
+if __name__ == "__main__":
+    src = pathlib.Path(sys.argv[1]).read_text()
+    print(json.dumps(check(src), indent=2))

vysort-0.1.0/src/vysort/workers/match.py

@@ -0,0 +1,119 @@
+"""Brute-force the deployed internal-function layout: enumerate every layout
+reachable by the 0.3.4-0.3.7 call-graph topsort, reorder the source per
+candidate, stock-compile, and compare against the target runtime bytecode.
+
+A `prefix` status means the compiled runtime is a prefix of the target and
+the remainder is the appended immutable tail. In `single` mode (unaffected
+compiler versions: exactly one layout is reachable) the original source is
+compiled once and compared.
+
+Self-contained worker: runs in an ephemeral `uv run --with vyper==X` env,
+importing siblings from its own directory. argv: <source.vy>
+<runtime-hex-file> <mode: brute|single> [evm-version]; JSON on stdout,
+progress on stderr.
+"""
+
+import json
+import pathlib
+import sys
+
+import vyper
+from check import capture_call_graph, enumerate_layouts, simulate
+from recover import recover_layout
+from reorder import reorder
+
+EVM_VERSION = sys.argv[4] if len(sys.argv) > 4 else None
+
+
+def compile_runtime(src: str) -> bytes:
+    kwargs = {"evm_version": EVM_VERSION} if EVM_VERSION else {}
+    out = vyper.compile_code(src, output_formats=["bytecode_runtime"], **kwargs)
+    return bytes.fromhex(out["bytecode_runtime"].removeprefix("0x"))
+
+
+def compare(target: bytes, runtime: bytes):
+    if runtime == target:
+        return {"status": "exact"}
+    if len(target) > len(runtime) and target.startswith(runtime):
+        return {"status": "prefix", "immutable_tail_bytes": len(target) - len(runtime)}
+    return None
+
+
+def match_single(src: str, target: bytes) -> dict:
+    verdict = compare(target, compile_runtime(src)) or {"status": "no_match"}
+    out = {**verdict, "attempts": 1, "reachable_layouts": 1}
+    if verdict["status"] != "no_match":
+        out["source"] = src
+    return out
+
+
+def match_brute(src: str, target: bytes) -> dict:
+    # layout recovery first: decode the deployed order straight off the
+    # bytecode, then one reordered stock compile to verify — 2 compiles
+    # total regardless of how many layouts are reachable
+    # a decode crash must not kill the worker — brute force still rescues
+    # 25 of the 26 at-risk mainnet runtimes if recovery hits an unknown
+    # edge case (the decode found new ones in both validation rounds)
+    try:
+        rec = recover_layout(src, target, EVM_VERSION)
+    except Exception as e:
+        rec = {"status": f"recover_error: {type(e).__name__}: {e}"}
+    if rec["status"] == "unsupported_minimal_proxy":
+        return rec
+    if rec["status"] == "decoded":
+        reordered = reorder(src, rec["proof_order"])
+        verdict = compare(target, compile_runtime(reordered))
+        if verdict:
+            return {
+                **verdict,
+                "method": "recovery",
+                "layout": rec["proof_order"],
+                "compiles": 2,
+                "source": reordered,
+            }
+        print("recovered order did not verify, falling back to brute force", file=sys.stderr)
+    else:
+        print(f"layout recovery unavailable ({rec['status']}), falling back to brute force", file=sys.stderr)
+
+    captured = capture_call_graph(src)
+    source_order = captured.pop("__source_order__")
+    internal = {n: v["internal"] for n, v in captured.items()}
+    base_callees = {n: v["callees"] for n, v in captured.items()}
+    env_layout = simulate(source_order, base_callees, internal)
+    layouts, exact, space = enumerate_layouts(source_order, base_callees, internal)
+    if not exact:
+        return {
+            "status": "layout_space_too_large",
+            "choice_space": space,
+            "sampled_layouts": len(layouts),
+        }
+    # the environment's own layout is the most likely match, try it first
+    candidates = sorted(layouts, key=lambda la: (la != env_layout, la))
+    for i, layout in enumerate(candidates, 1):
+        print(f"candidate {i}/{len(candidates)}: {list(layout)}", file=sys.stderr)
+        reordered = reorder(src, list(layout))
+        verdict = compare(target, compile_runtime(reordered))
+        if verdict:
+            return {
+                **verdict,
+                "method": "brute",
+                "layout": list(layout),
+                "attempts": i,
+                "reachable_layouts": len(candidates),
+                "source": reordered,
+            }
+    return {
+        "status": "no_match",
+        "method": "brute",
+        "attempts": len(candidates),
+        "reachable_layouts": len(candidates),
+    }
+
+
+if __name__ == "__main__":
+    src = pathlib.Path(sys.argv[1]).read_text()
+    hex_str = "".join(pathlib.Path(sys.argv[2]).read_text().split()).removeprefix("0x")
+    target = bytes.fromhex(hex_str)
+    mode = sys.argv[3]
+    out = match_single(src, target) if mode == "single" else match_brute(src, target)
+    print(json.dumps(out, indent=2))

vysort-0.1.0/src/vysort/workers/preflight.py

@@ -0,0 +1,42 @@
+"""Compile the exact standard-json payload that will be submitted to the
+verifier — through vyper's own std-json entry point, the same path the
+official binary takes — and compare the produced deployedBytecode against
+the target runtime. Catches anything that diverges between the in-process
+match verification and the submitted payload: settings serialization,
+std-json defaults, identifier resolution.
+
+Self-contained worker: runs in an ephemeral `uv run --with vyper==X` env.
+argv: <stdjson-file> <runtime-hex-file> <contract-identifier>; JSON on
+stdout.
+"""
+
+import json
+import pathlib
+import sys
+
+from vyper.cli.vyper_json import compile_json, exc_handler_to_dict
+
+
+def preflight(input_json: dict, target: bytes, identifier: str) -> dict:
+    filename, name = identifier.rsplit(":", 1)
+    output = compile_json(input_json, exc_handler=exc_handler_to_dict)
+    errors = [e for e in output.get("errors", []) if e.get("severity") == "error"]
+    if errors:
+        return {"status": "compile_error", "errors": [e.get("message") for e in errors]}
+    evm = output["contracts"][filename][name]["evm"]
+    runtime = bytes.fromhex(evm["deployedBytecode"]["object"].removeprefix("0x"))
+    if runtime == target:
+        return {"status": "exact"}
+    if len(target) > len(runtime) and target.startswith(runtime):
+        return {"status": "prefix", "immutable_tail_bytes": len(target) - len(runtime)}
+    n = next(
+        (i for i, (x, y) in enumerate(zip(target, runtime)) if x != y),
+        min(len(target), len(runtime)),
+    )
+    return {"status": "no_match", "common_prefix_bytes": n, "compiled_bytes": len(runtime)}
+
+
+if __name__ == "__main__":
+    input_json = json.loads(pathlib.Path(sys.argv[1]).read_text())
+    hex_str = "".join(pathlib.Path(sys.argv[2]).read_text().split()).removeprefix("0x")
+    print(json.dumps(preflight(input_json, bytes.fromhex(hex_str), sys.argv[3]), indent=2))

vysort-0.1.0/src/vysort/workers/recover.py

@@ -0,0 +1,178 @@
+"""Recover the deployed internal-function layout from on-chain runtime
+bytecode in 1 instrumented compile plus an offline decode (no search):
+
+  1. instrumented compile: walk the runtime assembly to get each internal
+     function's section boundaries (entry-label pcs) and all code-symbol
+     PUSH2 operand positions, masking only operands whose target lies in
+     the movable internal region;
+  2. decode the deployed section order by masked comparison (no compiles).
+
+Section lengths are order-invariant, so the decode terminates with the full
+deployed permutation regardless of how many layouts are reachable.
+
+Handles: shared revert/postamble after the last internal function (excluded
+from the movable region), cleanup-only internals (no emitted main label —
+anchored to the host section containing their cleanup pc and re-inserted
+into the proof order), and EIP-1167 minimal proxies (classified, not
+decoded).
+
+Self-contained worker module: runs in an ephemeral `uv run --with vyper==X`
+env; used by match.py, not invoked directly.
+"""
+
+import re
+
+import vyper.evm.opcodes as evm_ops
+from vyper.compiler.phases import CompilerData
+from vyper.ir.compile_ir import is_mem_sym, is_ofst, is_symbol
+
+EIP1167_RE = re.compile(r"^363d3d373d3d3d363d73([0-9a-f]{40})5af43d82803e903d91602b57fd5bf3$")
+
+
+def scan(assembly):
+    """Replicate assembly_to_evm's pc walk.
+
+    Returns (labels: symbol -> pc, refs: [(operand_pc, symbol)], code_end).
+    """
+    pc, labels, refs = 0, {}, []
+    for i, item in enumerate(assembly):
+        if item == "DEBUG":
+            continue
+        if is_symbol(item):
+            if assembly[i + 1] in ("JUMPDEST", "BLANK"):
+                labels[item] = pc
+            else:
+                refs.append((pc + 1, item))
+                pc += 3
+        elif is_mem_sym(item):
+            raise AssertionError("mem symbol in runtime assembly")
+        elif is_ofst(item):
+            pc -= 1  # matches upstream: following symbol overlaps by one byte
+        elif item == "BLANK" or (isinstance(item, str) and item.startswith("_DEPLOY_MEM_OFST_")):
+            pass
+        elif isinstance(item, list):
+            raise AssertionError("subcode in runtime assembly")
+        else:
+            pc += 1
+    return labels, refs, pc
+
+
+def recover_layout(src: str, onchain: bytes, evm_version=None) -> dict:
+    """One instrumented compile + offline decode of the deployed order.
+
+    Returns {"status": "decoded", "proof_order": [...], ...} on success;
+    other statuses: unsupported_minimal_proxy, too_few_sections,
+    decode_failed."""
+    m = EIP1167_RE.match(onchain.hex())
+    if m:
+        return {"status": "unsupported_minimal_proxy", "implementation": "0x" + m.group(1)}
+
+    if evm_version:
+        # 0.3.x anchors the evm version in a module global (no kwarg on CompilerData)
+        evm_ops.active_evm_version = evm_ops.EVM_VERSIONS[evm_version]
+    data = CompilerData(src)
+    bc = data.bytecode_runtime
+    internal_sigs = {n: s for n, s in data.function_signatures.items() if s.internal}
+    labels, refs, code_end = scan(data.assembly_runtime)
+
+    starts, missing = {}, []
+    for name, sig in internal_sigs.items():
+        main_pc = labels.get("_sym_" + sig.internal_function_label)
+        if main_pc is not None:
+            starts[name] = main_pc
+        else:
+            missing.append((name, sig))
+
+    if len(starts) < 2:
+        # nothing movable (cleanup-only internals can't reorder on their own)
+        return {"status": "too_few_sections", "internal_sections": len(starts)}
+
+    order = sorted(starts, key=starts.get)
+    movable_start = starts[order[0]]
+    # the shared revert/postamble after the last function is NOT movable:
+    # bound the last section at the first _sym_revert* label past its start
+    revert_pcs = [
+        pc for s, pc in labels.items() if s.startswith("_sym_revert") and pc > starts[order[-1]]
+    ]
+    movable_end = min(revert_pcs) if revert_pcs else code_end
+    bounds = [starts[f] for f in order] + [movable_end]
+    sections = {f: (bounds[i], bounds[i + 1]) for i, f in enumerate(order)}
+
+    # cleanup-only internals: no emitted main label; their remnant code lives
+    # inside another function's section. Anchor each to that host section so
+    # the proof order can re-insert it right after its host. Internals with
+    # neither label are never emitted in the runtime (unreachable from any
+    # external function, e.g. dead code or constructor-only) — their source
+    # position cannot affect the runtime layout, so they are left out of the
+    # proof order; the final byte-exact comparison validates this.
+    skipped, absent = [], []
+    for name, sig in missing:
+        cleanup_pc = labels.get("_sym_" + sig.exit_sequence_label)
+        host = None
+        if cleanup_pc is not None:
+            host = next((f for f, (lo, hi) in sections.items() if lo <= cleanup_pc < hi), None)
+        if cleanup_pc is None:
+            absent.append(name)
+        elif host is None:
+            return {
+                "status": "decode_failed",
+                "reason": f"cleanup label of {name} at pc {cleanup_pc} falls outside every internal section",
+            }
+        else:
+            skipped.append({"function": name, "attach_after": host, "cleanup_pc": cleanup_pc})
+
+    # mask only operands whose target label sits inside the movable region;
+    # refs to fixed targets (external section, postamble, code_end) must match
+    # as-is. _sym_code_end is the one symbol the assembler defines outside its
+    # main loop; it resolves to code_end, which is layout-invariant.
+    labels.setdefault("_sym_code_end", code_end)
+    mask = set()
+    for operand_pc, sym in refs:
+        target = labels.get(sym)
+        if target is None:
+            raise AssertionError(f"reference to undefined symbol {sym}")
+        if movable_start <= target < movable_end:
+            mask.update((operand_pc, operand_pc + 1))
+
+    def eq_masked(off, lo, hi):
+        if off + (hi - lo) > len(onchain):
+            return False
+        return all(onchain[off + i - lo] == bc[i] for i in range(lo, hi) if i not in mask)
+
+    external_ok = eq_masked(0, 0, movable_start)
+
+    cursor, deployed, remaining = movable_start, [], set(order)
+    while remaining:
+        for f in sorted(remaining):
+            lo, hi = sections[f]
+            if eq_masked(cursor, lo, hi):
+                deployed.append(f)
+                remaining.discard(f)
+                cursor += hi - lo
+                break
+        else:
+            return {
+                "status": "decode_failed",
+                "reason": f"no section matches at offset {cursor}",
+                "recovered_so_far": deployed,
+                "external_region_ok": external_ok,
+            }
+
+    proof_order = []
+    for f in deployed:
+        proof_order.append(f)
+        proof_order.extend(
+            s["function"]
+            for s in sorted(skipped, key=lambda s: s["cleanup_pc"])
+            if s["attach_after"] == f
+        )
+
+    return {
+        "status": "decoded",
+        "recovered_order": deployed,
+        "proof_order": proof_order,
+        "skipped_internals": [s["function"] for s in skipped],
+        "absent_internals": absent,
+        "internal_sections": len(order),
+        "external_region_ok": external_ok,
+    }

vysort-0.1.0/src/vysort/workers/reorder.py

@@ -0,0 +1,59 @@
+"""Force a vanilla vyper 0.3.x compiler into a specific internal-function
+layout by reordering source definitions: move the targeted internal function
+defs to the front (before any other function def), ordered by the target
+layout. With internals defined before every caller, the topsort has zero
+decision points and the layout is deterministic in any environment.
+
+Uses vyper's own AST for exact function spans (handles multi-line signatures).
+
+Self-contained worker: runs in an ephemeral `uv run --with vyper==X` env and
+imports nothing from vysort. argv: <source.vy> <layout-json>; reordered
+source on stdout.
+"""
+
+import json
+import pathlib
+import sys
+
+import vyper.ast as vy_ast
+
+
+def reorder(src: str, layout: list) -> str:
+    lines = src.splitlines(keepends=True)
+    mod = vy_ast.parse_to_ast(src)
+    spans = {}  # name -> (start_line0, end_line0_exclusive)
+    first_def_start = None
+    for f in mod.get_children(vy_ast.FunctionDef):
+        start = min([d.lineno for d in f.decorator_list] + [f.lineno]) - 1
+        # include contiguous leading comment lines so moved functions keep
+        # their banner comments/NatSpec (cosmetic; bytecode is unaffected)
+        while start > 0 and lines[start - 1].strip().startswith("#"):
+            start -= 1
+        end = f.end_lineno
+        spans[f.name] = (start, end)
+        if first_def_start is None or start < first_def_start:
+            first_def_start = start
+    missing = [n for n in layout if n not in spans]
+    if missing:
+        raise SystemExit(f"functions not found in source: {missing}")
+
+    moved = set()
+    for n in layout:
+        moved.update(range(*spans[n]))
+
+    out = []
+    for i, line in enumerate(lines):
+        if i == first_def_start:
+            for n in layout:
+                s, e = spans[n]
+                out.extend(lines[s:e])
+                out.append("\n\n")
+        if i not in moved:
+            out.append(line)
+    return "".join(out)
+
+
+if __name__ == "__main__":
+    src = pathlib.Path(sys.argv[1]).read_text()
+    layout = json.loads(sys.argv[2])
+    sys.stdout.write(reorder(src, layout))