sememe 0.4.21 → 0.4.23

This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
package/README.md CHANGED
@@ -69,7 +69,7 @@ What this changes about the thesis, stated plainly: sememe's decode path is stro
69
69
 
70
70
  **HADRo's 195 refusals were NOT source lag — stored discriminators are stored at ANY length (fixed and measured 2026-11-12, `tools/measure-hadro-short-discriminator.mjs` -> `research/hadro-short-discriminator-2026-11-12.json`, pinned by `test/hadro-short-discriminator.test.mjs` 7 tests + `test/hadro-short-discriminator-measurement.test.mjs` 5 tests).** The ecosystem measurement had labelled HADRo (`HADRo…VDQ8`) a "total source-lag face" at 0/195. It was wrong, and the PULS V two-tree measurement (`git archive c304e80` for the pre-fix tree, a byte-identical probe in both) proved why: the program's published IDL stores **38 one-byte instruction discriminators** — a real, chain-accepted wire table (unique values 0x00..0x26) — and `withDerivedDiscriminators` treated only 8-byte arrays as "stored", so every stored byte was OVERWRITTEN with Anchor's 8-byte sighash, against that module's own header rule ("stored discriminators are never changed"). The replaced table matches no payload: pre-fix 0/195 decoded; post-fix **123/195 decode through the product path** (51 full byte-exact, 72 `setCurve` described-exact with reported trailing bytes), and the remaining 72 (`setRiskCurve`) stay **honestly refused and NAMED** — that IDL declares a vector length the wire bytes do not carry. The same `length === 8` assumption in `src/union.mjs`'s rename-evidence map was fixed the same way (stored means any non-empty byte array). The fix is surgical and the measurement pins that: pump.fun (400/842, 249 byte-exact), pfeeU (5/464) and Eternal (227/242 byte-exact) verdicts are BYTE-IDENTICAL between the trees; Eternal's 76 stored 1-byte EVENT discriminators are now preserved too (the event dispatch path reads 8-byte heads, so that preservation changes no decode in this window — stated, not smoothed). Honest limits ride in the artifact; the unit face flips `false -> true` on both instruction and event preservation, and the store face `0/38 -> 38/38`.
71
71
 
72
- The stranger-facing truth this pins down: **the 1,000-transaction decode is one command, offline, and fails loudly.** What it does not prove is that an outside person has run it — and that is now **measured, not asserted** (`tools/measure-foreign-run.mjs` -> `research/foreign-run-2026-11-05.json`): verdict `not-distributed-not-consumed`, i.e. zero non-owner signals on every channel that was read — 0 non-owner stars/forks/issues/PRs on this repo, 0 non-owner comments on the two outbound threads (`step-finance-oss/step-ingestooor#1` and `jup-ag/rfq-webhook-toolkit#96`, both open), 0 foreign-authored threads containing this project's slug. The first external run therefore remains the open gate. The measurement has limits and names them: the broad mention sweep covers 92 of 93 results (GitHub's search index hides one result in every ordering — measured, not assumed), a registry `404` for the bare name `sememe` on npm/crates.io/PyPI means **no install channel is published yet**, never "nobody used it", and the code-index absence is only claimed for the indexed part of GitHub (the sweep's self-reference control passes). Reproduce with `npm test` (includes `test/stranger-decode.test.mjs`, which runs this tool against both fixtures and falsifies it three ways: tampered payload, missing store, and a mutation of the pass/fail logic). The funnel a stranger walks before that first run is measured separately (2026-11-05, `tools/measure-stranger-funnel.mjs` -> `research/stranger-funnel-2026-11-05.json`): a fresh GitHub clone ran both quickstart commands verbatim (1,089 and 6,337 payloads, all decoded byte-exactly, exit 0), the published tag `v0.3.0` installed into an empty app with no npm account, and its entry decoded 1,119/1,119 real payloads byte-exactly from `node_modules/sememe` — the harness (tools/test/research) does not travel into the install. The quickstart block itself is pinned by `test/quickstart-funnel.test.mjs`, which caught and fixed one defect on 2026-11-05: the documented output line was the pre-2026-11-04 schema (missing `rpcEncoding`, `chainParsedInstructions`, `missingBytes`) — a stranger comparing her terminal to the README saw two different products. The README now documents the line the tool actually prints, and a hand-edited documented line goes red.
72
+ The stranger-facing truth this pins down: **the 1,000-transaction decode is one command, offline, and fails loudly.** What it does not prove is that an outside person has run it — and that is now **measured, not asserted** (`tools/measure-foreign-run.mjs` -> `research/foreign-run-2026-11-05.json`): verdict `not-distributed-not-consumed`, i.e. zero non-owner signals on every channel that was read — 0 non-owner stars/forks/issues/PRs on this repo, 0 non-owner comments on the two outbound threads (`step-finance-oss/step-ingestooor#1` and `jup-ag/rfq-webhook-toolkit#96`, both open), 0 foreign-authored threads containing this project's slug. The first external run therefore remains the open gate. The measurement has limits and names them: the broad mention sweep covers 92 of 93 results (GitHub's search index hides one result in every ordering — measured, not assumed), a registry `404` for the bare name `sememe` on npm/crates.io/PyPI meant **no install channel was published yet** at that time, never "nobody used it" — npm published the name on 2026-11-12 (crates.io/PyPI still have no `sememe`; both are aliasing checks, not consumption evidence), and the code-index absence is only claimed for the indexed part of GitHub (the sweep's self-reference control passes). Reproduce with `npm test` (includes `test/stranger-decode.test.mjs`, which runs this tool against both fixtures and falsifies it three ways: tampered payload, missing store, and a mutation of the pass/fail logic). The funnel a stranger walks before that first run is measured separately (2026-11-05, `tools/measure-stranger-funnel.mjs` -> `research/stranger-funnel-2026-11-05.json`): a fresh GitHub clone ran both quickstart commands verbatim (1,089 and 6,337 payloads, all decoded byte-exactly, exit 0), the published tag `v0.3.0` installed into an empty app with no npm account, and its entry decoded 1,119/1,119 real payloads byte-exactly from `node_modules/sememe` — the harness (tools/test/research) does not travel into the install. The quickstart block itself is pinned by `test/quickstart-funnel.test.mjs`, which caught and fixed one defect on 2026-11-05: the documented output line was the pre-2026-11-04 schema (missing `rpcEncoding`, `chainParsedInstructions`, `missingBytes`) — a stranger comparing her terminal to the README saw two different products. The README now documents the line the tool actually prints, and a hand-edited documented line goes red. Re-measured 2026-11-13 after the registry channel opened (`research/foreign-run-2026-11-12.json`): the registry check reads `npm: found` for the bare name while the verdict stays `not-distributed-not-consumed` — publish is not consumption, and the first external run remains the open gate.
73
73
 
74
74
  ## Your own transactions: raw RPC responses (the shape you already hold)
75
75
 
@@ -132,7 +132,7 @@ sememe is a local, content-addressed schema store with a verification contract:
132
132
  ## Install (as a consumer, not a contributor)
133
133
 
134
134
  ```bash
135
- npm install github:auroraxo/sememe#v0.4.21 # tagged release; no npm account needed
135
+ npm install github:auroraxo/sememe#v0.4.23 # tagged release; no npm account needed
136
136
  ```
137
137
 
138
138
  ```js
@@ -151,14 +151,14 @@ The **library** refuses the same way: importing `sememe/pull` or `sememe/gap` wi
151
151
  **From an install (not a clone), the CLI is `npx sememe …`** — npm exec resolves the package's own `bin` from `node_modules`; the bare command `sememe` is not on your `PATH` after a plain `npm install`, and `node bin/sememe.mjs` works only inside a clone of the repository (both measured 2026-11-05 from a real tarball install). The consumer journey from the install, verbatim:
152
152
 
153
153
  ```bash
154
- npm install github:auroraxo/sememe#v0.4.21 @solana/web3.js # acquisition needs the peer
154
+ npm install github:auroraxo/sememe#v0.4.23 @solana/web3.js # acquisition needs the peer
155
155
  npx sememe pull whirLbMiicVdio4qvUfM5KAg6Ct8VwpYzGff3uctyCc --store ./store
156
156
  npx sememe decode whirLbMiicVdio4qvUfM5KAg6Ct8VwpYzGff3uctyCc 2b04ed0b1ac91e62a40d0100000000000100000000000000af331ba8327fbb35b1c4feff00000000010001010000000601 --store ./store
157
157
  ```
158
158
 
159
159
  `./store` is the local content-addressed store the error `no sememe store at ./store: run "sememe pull" first` tells you to create — the `pull` above is that creation step; the vendored programs (System, ComputeBudget, SPL Token, Token-2022, Raydium AMM v4, SPL Associated Token Account) need no store at all: `npx sememe decode-native 11111111111111111111111111111111 <payload>` answers offline from the pinned upstream tables. If you ever point the decoder at a store directory that does not exist yet, the refusal names the fix; `npx sememe` is the form of that fix that runs where you are. Measured 2026-11-05 from the packed tarball of this tree: install (zero deps) → `npx sememe pull` (live mainnet) → `npx sememe decode` (byte-exact, live-pulled object byte-identical to the fixture store object `0b886e37…660f`).
160
160
 
161
- Publishing to the npm registry itself is deliberately not done: the files allowlist already ships none of this repository's tests, fixtures or qualification harness (pinned by `test/package.test.mjs`), and no registry name squatting is part of the plan. The git tag is the release artifact; `test/package.test.mjs` (the contract above), `tools/e2e-package-consumer.mjs` (a real install into an empty directory) and `test/install-surface.test.mjs` (the shipped README's install commands run verbatim from the packed tarball) keep it honest.
161
+ Publishing to the npm registry changed on 2026-11-12: the `sememe` name is now live on registry.npmjs.org (first shipped as 0.4.21; the measured registry install of `sememe@0.4.22` into an empty app decoded the Whirlpool fixture payload `swap_v2`, `roundTripByteExact: true`). `npm install sememe` and `npm install github:auroraxo/sememe#v0.4.23` deliver the same product files from the same code: the files allowlist already ships none of this repository's tests, fixtures or qualification harness (pinned by `test/package.test.mjs`), so the git tag remains the release artifact — `test/package.test.mjs` (the contract above), `tools/e2e-package-consumer.mjs` (a real install into an empty directory) and `test/install-surface.test.mjs` (the shipped README's install commands run verbatim from the packed tarball) keep both channels honest.
162
162
 
163
163
  ## Working now: pin the answers (`sememe lock` + `sememe verify`, 2026-11-05)
164
164
 
@@ -271,7 +271,8 @@ freshness against the chain; `scope: "extra"` tells you *where* the damage
271
271
  is, not *who* wrote it; and the reported extra is an unpinned program, so a
272
272
  store can still gain a program the lock has never seen (by design — that is
273
273
  the minimums rule). No third party has run this; there are 0 external users,
274
- 0 stars, and `npm publish` remains credentials-blocked. The published
274
+ 0 stars (`npm publish` was credentials-blocked until 2026-11-12, when the
275
+ registry name shipped — see the Install section). The published
275
276
  `v0.4.10` tag keeps this defect (a tag is immutable); `v0.4.11` carries
276
277
  the fix.
277
278
 
@@ -2570,6 +2571,166 @@ way shank was. What this fix guarantees is that until that happens, the
2570
2571
  interface is refused by name at every boundary and never crashes the codec
2571
2572
  or pretends to be decodable.
2572
2573
 
2574
+ ## The product SPEAKS the dialect it named: the raw-LE engine is wired (2026-11-12)
2575
+
2576
+ PULS X named the refusal ("naming a dialect is not speaking it") and PULS Y
2577
+ built the speaker — an engine for exactly the grammar the diabloarb file
2578
+ DECLARES (`encoding: "little-endian; u8/u16/u32/u64 unsigned, i64 signed; no
2579
+ padding; no length prefixes except the u8 counters named in each header"`):
2580
+ u8-tag dispatch, dict `types` with declared fixed sizes, counter-prefixed
2581
+ vectors (`Base[n_bases]`, `u8[n_hops]`), one byte-decided optional trailing
2582
+ field. 102/102 committed payloads decoded byte-exactly, and the product
2583
+ boundary stayed untouched — deliberately: capability first, the wiring with
2584
+ the pins it rewrites.
2585
+
2586
+ **This pulse is the wiring** (the card's own declared next step):
2587
+
2588
+ - `src/codec.mjs` routes every decode/encode of a file OUTSIDE the Anchor
2589
+ shape but INSIDE the engine grammar through `src/rawle.mjs`, before the
2590
+ shape gate. The result carries the codec's own fields (`discriminator` is
2591
+ the dispatch prefix in hex; `dialect: 'raw-le'`), and every engine refusal
2592
+ is translated into `CodecError` — one refusal family at the product
2593
+ boundary, never a `RawLeError` leak, never a crash.
2594
+ - `src/dialect.mjs` splits the question "can anything read this file?" from
2595
+ "can the Anchor codec read it?": `unreadableDialectReason`/`assertReadableDialect`
2596
+ pass an engine-readable file and keep the exact named refusal (code
2597
+ `unsupported-idl-dialect`) for everything else; `idlDialect` reports
2598
+ `'raw-le'` for a wired dialect.
2599
+ - The store gates (`src/adopt.mjs` write path, `src/store-read.mjs` read
2600
+ path) use the readable-dialect guard: a raw-LE object adopts under its
2601
+ own program id, reads back, and the CLI loop `sememe adopt` → `sememe
2602
+ decode` works end-to-end offline (measured 2/2). Mislabelling under a
2603
+ foreign program id stays refused via `program_id` — the wiring opens the
2604
+ dialect gate, not the re-labelling gate.
2605
+ - The derivation (`src/discriminators.mjs`) passes the file WITHOUT filling
2606
+ any Anchor sighash table (the PULS X fill defect stays dead — measured
2607
+ filled {instructions: 0, events: 0}); the union (`src/union.mjs`)
2608
+ decodes through the engine like through any source.
2609
+
2610
+ Measured two-tree, byte-identical probe
2611
+ ([`research/rawle-product-2026-11-12.json`](research/rawle-product-2026-11-12.json),
2612
+ pinned in `test/rawle-product-measurement.test.mjs` with a full
2613
+ gh-gated re-derivation): prefix tree = the PULS Y engine commit `05e6643`
2614
+ (engine present, product refuses — 0/102, store loop closed), this tree =
2615
+ 102/102 decoded AND 102/102 byte-exact re-encoded at `decodeInstructionData`,
2616
+ 5/5 product-level refusals as named `CodecError` faces (unknown tag,
2617
+ truncated payload, trailing bytes, unknown instruction on encode, missing
2618
+ argument on encode), derivation passes with NO table, adopt/read/union open,
2619
+ CLI store loop 2/2, Whirlpool control unchanged, a file that defeats BOTH
2620
+ engines keeps the exact PULS X named refusal at codec AND union. 24
2621
+ falsification counts, all zero. The PULS X measurement
2622
+ ([`research/dialect-named-refusal-2026-11-12.json`](research/dialect-named-refusal-2026-11-12.json))
2623
+ was re-derived in its invariant form: its delta now reads crash → decodes
2624
+ through the engine, fill {4,0} → {0,0} — the no-crash, no-fill, no-mislabel
2625
+ invariants pinned on every payload. The PULS Y artifact stays the frozen
2626
+ pre-wiring baseline; its tool now exits 3 NO VERDICT in a wired tree BY
2627
+ DESIGN (stated in the tool header).
2628
+
2629
+ **Tests: 1 new pin file** (`test/rawle-product-measurement.test.mjs`, 12
2630
+ tests) plus the bent faces in `test/dialect-named-refusal.test.mjs` (D1b),
2631
+ `test/rawle-decoder.test.mjs` (G1) and
2632
+ `test/dialect-named-refusal-measurement.test.mjs`. Suite 912 → **933: 929
2633
+ pass / 4 named network skips / 0 fail**. Product code changed; the pin is
2634
+ the next release.
2635
+
2636
+ ## The wiring SHIPS in v0.4.22 — and the ecosystem number it moves is +102 for the user who adopts, not the shipped store (2026-11-12)
2637
+
2638
+ This release carries the raw-LE wiring above to the install channel: the
2639
+ pinned tag now contains `src/rawle.mjs` and every wired boundary, so
2640
+ `npm install github:auroraxo/sememe#v0.4.22` degrades nobody (Whirlpool,
2641
+ shank, name-only anchor controls pin unchanged) and answers the diabloarb
2642
+ dialect where v0.4.21 refused it by name.
2643
+
2644
+ The wiring also prompted the card's declared coverage re-measurement, and
2645
+ the honest answer is TWO-HALVED (`tools/measure-diablo-adoption-coverage.mjs`
2646
+ → `research/diablo-adoption-coverage-2026-11-12.json`, pinned by
2647
+ `test/diablo-adoption-coverage.test.mjs`, 9 tests, gh-gated re-derivation
2648
+ included):
2649
+
2650
+ - **At the shipped store the headline does NOT move.** Re-running the
2651
+ committed 2026-11-11 fixture through the coverage tool on this tree
2652
+ reproduces the committed artifact exactly: **88.5%, DiabLokxis 0 decoded
2653
+ / 102 no-interface.** The wiring changed the codec, not the store, and
2654
+ the diabloarb copy cannot ship inside the store: `cryptodiablo/diabloarb`
2655
+ carries **no license** at the pin `f5005ac4` — live re-checked inside the
2656
+ pin test (GitHub license API 404 at the pin AND at HEAD, repo field
2657
+ null), not narrated. The bytes stay out of the repository and the
2658
+ package; `committedStoreModified: false` (readdir + per-file hash digest
2659
+ before/after).
2660
+ - **For the user who adopts, the 102 are real.** ONE `adopt` of the
2661
+ hash-gated first-party copy (canonical fingerprint
2662
+ `d6d4e8aa…` verified before anything runs) through the product's own
2663
+ write path, into a throwaway copy of the committed store, and the SAME
2664
+ report tool answers **102/102 decoded, 0 refused, 102 byte-exact**:
2665
+ headline **88.5% → 89.0%** of measured traffic, outside the watchlist
2666
+ **62.4% → 64.4%**. Every one of the 102 rides the **adopted** evidence
2667
+ column (`decodedVia.adopted +102`; `pulled` moves by exactly zero) — the
2668
+ same class split the pfeeU, manifest and archer measurements keep, so an
2669
+ adoption can never silently inflate the on-chain-pull number.
2670
+
2671
+ So "102 jetzt zählbar" was half true: countable **yes** — as adoption-path
2672
+ coverage, with a name and a byte-exact check — and **not** as a shipped
2673
+ headline. The acquisition gap at that program is a LICENSE gap, not a
2674
+ decoder gap: the one thing between the user and those 102 payloads is a
2675
+ licensed first-party copy, which is exactly what `sememe adopt` exists to
2676
+ carry while origin stays declared.
2677
+
2678
+ Suite after this release: **954 tests / 950 pass / 4 named network skips /
2679
+ 0 fail** (945 → 954: the 9 new pin tests). CI green on `main`
2680
+ (43287274405) and on the published tag v0.4.22 (43287274209); the release
2681
+ surface re-measured against that tag is clean (newest = latest = pinned,
2682
+ 0 defects — `research/release-surface-2026-11-12b.json`).
2683
+
2684
+ ## v0.4.23: struct-internal absent fields decode under the evidence gate — and the registry stays truthful (2026-11-13)
2685
+
2686
+ The absent-field policy (`absent: "report"`) only fired at TOP-LEVEL argument
2687
+ boundaries, so a payload that ends on the boundary of the LAST FIELD INSIDE a
2688
+ struct argument was refused under every policy. Measured on mainnet: Archer's
2689
+ legacy Swap form (tag `0f`, 19 bytes) omits `SwapParams.builderFeePpm`
2690
+ INSIDE `swapParams` and is chain-executed — witness `5Kpdai4oRG…`, slot
2691
+ 468261978, `err: null`, instruction bytes byte-identical to the payload
2692
+ (`research/archer-legacy-swap-witness-2026-11-12.json`; the pinned IDL's own
2693
+ `SwapParams` docs describe the shorter form as supported and read as a 0 fee,
2694
+ with the program's own `success` log line). One of the 83 payloads in
2695
+ `research/archer-second-source-2026-11-12.json` hit exactly this and v0.4.22
2696
+ refused it with `truncated input: need 4 byte(s) at offset 19`.
2697
+
2698
+ This release decodes it and re-encodes it byte-exactly: inside a named struct,
2699
+ under the same decode-only evidence gate (`ctx.__absentPolicy`), a boundary
2700
+ ending mid-struct makes the remaining fields ABSENT, named by their dotted path
2701
+ (`swapParams.builderFeePpm`), never invented; cuts INSIDE a field and boundary
2702
+ cuts inside vec/array elements still refuse; the strict default policy is
2703
+ untouched; `encodeInstructionData` accepts the dotted absent names (trailing
2704
+ run per struct). `verifiedShortForm` now matches short (sub-8-byte) shank
2705
+ discriminators by prefix — the rule the codec's instruction index already used
2706
+ — so the 1-byte Archer tag reaches the chain-verified short-form table. Pinned
2707
+ by `test/struct-absent-short-form.test.mjs` (10 tests: the falsification
2708
+ ladder is 6 red on the v0.4.22 tree, green here).
2709
+
2710
+ The registry channel is live and carries this release with the tag: the
2711
+ foreign-run sweep measured `npm: found` for the bare name with the verdict
2712
+ still `not-distributed-not-consumed`
2713
+ (`research/foreign-run-2026-11-12.json`) — publish is not consumption, and
2714
+ the first external run stays the open gate.
2715
+
2716
+ Install — both channels ship the same product files:
2717
+
2718
+ ```bash
2719
+ npm install sememe # registry
2720
+ npm install github:auroraxo/sememe#v0.4.23 # tagged release
2721
+ ```
2722
+
2723
+ Limits, stated plainly: the Archer interface still does not ship inside the
2724
+ store (no on-chain IDL; the Apache-2.0 first-party copy stays adopt-only and
2725
+ hash-gated — unchanged); enum- and vec-body boundary cuts still refuse until a
2726
+ chain witness names them (the rule stays evidence-gated; no speculative wire
2727
+ form is accepted); tags <= v0.4.22 keep the truncation refusal (a tag is
2728
+ immutable); one owner account, zero external users, no audit.
2729
+
2730
+ Suite for this release: **964 tests / 960 pass / 4 named network skips /
2731
+ 0 fail** (954 → 964: the 10 new pin tests). CI green on `main`
2732
+ (43332583023); release surface re-measured after publishing below.
2733
+
2573
2734
  ## The remaining 35 publish-nothing programs, classified against foreign sources (2026-11-04)
2574
2735
 
2575
2736
  With Lighthouse decoded under shank, exactly **35 programs (87 payloads)** remained
package/package.json CHANGED
@@ -1,6 +1,6 @@
1
1
  {
2
2
  "name": "sememe",
3
- "version": "0.4.21",
3
+ "version": "0.4.23",
4
4
  "description": "Local, fingerprint-verified Solana program interfaces: pull an IDL once, then decode instructions, accounts and events offline. Zero dependencies on the decode path. Ships derived wire tables for System, ComputeBudget, SPL Token, Token-2022 (incl. its token-metadata fallback), Raydium AMM v4 and the SPL Associated Token Account, pinned to upstream Rust.",
5
5
  "license": "MIT",
6
6
  "type": "module",
package/src/adopt.mjs CHANGED
@@ -39,7 +39,7 @@ import path from 'node:path';
39
39
  import { canonicalJson } from './canonical.mjs';
40
40
  import { writeOnce } from './store-write.mjs';
41
41
  import { originDigestOf, receiptDigestOf } from './receipt-digest.mjs';
42
- import { assertIdlShape, declaredProgram } from './dialect.mjs';
42
+ import { assertIdlShape, assertReadableDialect, declaredProgram } from './dialect.mjs';
43
43
 
44
44
  const sha256 = (data) => createHash('sha256').update(data).digest('hex');
45
45
 
@@ -117,12 +117,13 @@ export async function adopt(programInput, filePath, storeDir, { source = 'publis
117
117
  }
118
118
  throw Error(`${filePath} declares program ${declared}, not ${program} — refusing to re-label a foreign interface`);
119
119
  }
120
- // Gate 2 — the dialect gate runs at the store's WRITE path too: an
121
- // interface the codec cannot read (measured 2026-11-12: diabloarb's
122
- // tag/data dispatch with dict `types`) is refused HERE, not after
123
- // adoption when every decode crashes unnamed. Same reason text as the
124
- // codec's guard (src/dialect.mjs).
125
- assertIdlShape(idl, (reason) => Error(`${filePath}: ${reason}`));
120
+ // Gate 2 — the dialect gate at the store's WRITE path: a dialect NO
121
+ // engine of the product can read is refused HERE by name, before
122
+ // adoption can store an undecodable interface. SINCE THE PULS-Z WIRING
123
+ // a raw-LE file passes (the codec serves it from src/rawle.mjs),
124
+ // everything else keeps the named refusal (measured 2026-11-12:
125
+ // diabloarb's dict `types` was the face of that refusal).
126
+ assertReadableDialect(idl, (reason) => Error(`${filePath}: ${reason}`));
126
127
 
127
128
  const json = Buffer.from(canonicalJson(idl) + '\n');
128
129
  const fingerprint = sha256(json);
package/src/codec.mjs CHANGED
@@ -20,7 +20,8 @@
20
20
 
21
21
  import { encodeBase58, decodeBase58 } from './base58.mjs';
22
22
  import { isShankIdl, isShankDiscriminant, shankDiscriminatorBytes, SHANK_LEB128_TYPES } from './shank.mjs';
23
- import { assertIdlShape } from './dialect.mjs';
23
+ import { assertIdlShape, unsupportedIdlReason } from './dialect.mjs';
24
+ import { rawLeDialectReason, decodeRawLe, encodeRawLe, RawLeError } from './rawle.mjs';
24
25
 
25
26
  const MAX_DEPTH = 32;
26
27
  // Instruction data is bounded by transaction size; a hostile length prefix
@@ -364,8 +365,14 @@ function decodeAt(type, cur, ctx, path, depth) {
364
365
  throw new CodecError(`vector length ${length} needs ${length * minimum} byte(s) but ${cur.remaining} remain`, path);
365
366
  }
366
367
  const out = [];
367
- for (let index = 0; index < length; index++) {
368
- out.push(decodeAt(type.vec, cur, ctx, `${path}[${index}]`, depth + 1));
368
+ const insideCollection = ctx.__insideCollection;
369
+ ctx.__insideCollection = true;
370
+ try {
371
+ for (let index = 0; index < length; index++) {
372
+ out.push(decodeAt(type.vec, cur, ctx, `${path}[${index}]`, depth + 1));
373
+ }
374
+ } finally {
375
+ ctx.__insideCollection = insideCollection;
369
376
  }
370
377
  return out;
371
378
  }
@@ -427,8 +434,40 @@ function decodeAt(type, cur, ctx, path, depth) {
427
434
  throw new CodecError('malformed struct field', path);
428
435
  }
429
436
  const out = {};
430
- for (const field of fields) {
431
- assign(out, field.name, decodeAt(field.type, cur, ctx, `${path}.${field.name}`, depth + 1), path);
437
+ for (const [index, field] of fields.entries()) {
438
+ // The payload ends exactly where this struct's next field would begin.
439
+ // Under the caller-risk 'report' policy (reached only through
440
+ // ctx.__absentPolicy — decodeInstructionData sets it, and nothing else
441
+ // in the codec opts in), the rest of THIS struct is ABSENT, not
442
+ // truncated: a field boundary is the only place a payload can end and
443
+ // still carry a complete value for what came before. The marker names
444
+ // the missing fields by their path RELATIVE TO THE ARGUMENT (e.g.
445
+ // builderFeePpm inside swapParams), so decodeInstructionData can report
446
+ // `swapParams.builderFeePpm` and tell a struct-internal absence from a
447
+ // missing top-level argument. Under the strict default this branch
448
+ // never engages and the field decode refuses as truncated.
449
+ if (ctx.__absentPolicy === 'report' && cur.remaining === 0) {
450
+ if (ctx.__insideCollection) {
451
+ // A struct inside a vec/array: the wire stopped inside a repeated
452
+ // element, and the element INDEX is not recoverable from the
453
+ // bytes. Naming an absence there would be a guess, so it refuses
454
+ // like a mid-field cut until a witness says what to call it.
455
+ throw new CodecError(
456
+ `truncated input: the payload ends inside a repeated struct field (${path})`,
457
+ path);
458
+ }
459
+ const missing = fields.slice(index).map((rest) => rest.name);
460
+ ctx.__absentCollected?.push(...missing.map((name) => [...(ctx.__absentTrail ?? []), name].join('.')));
461
+ out.__absentFields = missing;
462
+ return out;
463
+ }
464
+ const trail = ctx.__absentTrail;
465
+ trail?.push(field.name);
466
+ try {
467
+ assign(out, field.name, decodeAt(field.type, cur, ctx, `${path}.${field.name}`, depth + 1), path);
468
+ } finally {
469
+ trail?.pop();
470
+ }
432
471
  }
433
472
  return out;
434
473
  }
@@ -444,6 +483,10 @@ function decodeAt(type, cur, ctx, path, depth) {
444
483
  if (!shape) throw new CodecError(`malformed variant ${JSON.stringify(variant.name)}`, path);
445
484
  const out = { __kind: variant.name };
446
485
  if (shape.kind === 'unit') return out;
486
+ // Note: a payload cut inside an enum variant's fields refuses even under
487
+ // the absent:'report' policy — the struct-boundary absence rule covers
488
+ // struct arguments (the shape every measured witness used), not variant
489
+ // bodies. Widening it needs its own chain witness first.
447
490
  if (shape.kind === 'tuple') {
448
491
  out.fields = shape.fields.map((fieldType, position) =>
449
492
  decodeAt(fieldType, cur, ctx, `${path}.fields[${position}]`, depth + 1));
@@ -460,14 +503,46 @@ function decodeAt(type, cur, ctx, path, depth) {
460
503
 
461
504
  function decodeArray(inner, length, cur, ctx, path, depth) {
462
505
  const out = [];
463
- for (let index = 0; index < length; index++) {
464
- out.push(decodeAt(inner, cur, ctx, `${path}[${index}]`, depth + 1));
506
+ const insideCollection = ctx.__insideCollection;
507
+ ctx.__insideCollection = true;
508
+ try {
509
+ for (let index = 0; index < length; index++) {
510
+ out.push(decodeAt(inner, cur, ctx, `${path}[${index}]`, depth + 1));
511
+ }
512
+ } finally {
513
+ ctx.__insideCollection = insideCollection;
465
514
  }
466
515
  return out;
467
516
  }
468
517
 
469
518
  // --- encoder ----------------------------------------------------------------
470
519
 
520
+ /**
521
+ * Struct fields of a type that an absent PATH may descend through. Only
522
+ * named structs qualify: a dotted absent name like swapParams.builderFeePpm
523
+ * describes where the wire stopped, and only a named struct's fields can
524
+ * name that place. Anything else (tuple struct, enum, array, primitive) is a
525
+ * refusal — inventing an absence inside a shape that cannot name its fields
526
+ * would be exactly the guesswork the policy exists to avoid.
527
+ */
528
+ function structFieldsForAbsentPath(type, ctx, path) {
529
+ let current = type;
530
+ if (current && typeof current === 'object' && 'defined' in current) {
531
+ const name = typeof current.defined === 'string' ? current.defined : current.defined?.name;
532
+ const typedef = ctx.types.get(name);
533
+ if (!typedef) throw new CodecError(`absent path crosses unknown type ${JSON.stringify(name)}`, path);
534
+ current = typedef.type;
535
+ }
536
+ if (current && typeof current === 'object' && current.kind === 'type' && 'type' in current) {
537
+ current = current.type;
538
+ }
539
+ const fields = structShape(current);
540
+ if (!fields || !isNamedStructFields(fields)) {
541
+ throw new CodecError(`absent path crosses a type that is not a named struct`, path);
542
+ }
543
+ return fields;
544
+ }
545
+
471
546
  function encodeAt(type, value, ctx, path, depth) {
472
547
  if (depth > MAX_DEPTH) throw new CodecError('type nesting too deep (recursive type?)', path);
473
548
 
@@ -561,8 +636,31 @@ function encodeAt(type, value, ctx, path, depth) {
561
636
  const expected = new Set(fields.map((field) => field.name));
562
637
  const unknown = Object.keys(value).filter((key) => !expected.has(key));
563
638
  if (unknown.length) throw new CodecError(`unexpected field(s): ${unknown.join(', ')}`, path);
639
+ // Struct-internal absent fields: encodeInstructionData resolves dotted
640
+ // absent names (swapParams.builderFeePpm) to the VALUE object that owns
641
+ // the field and records the omission on ctx. Same rule as the top level:
642
+ // within this struct only a trailing run may be omitted, every omitted
643
+ // name must be declared, and what remains must still carry a value —
644
+ // byte-exactness is the only evidence this encoder offers.
645
+ const skip = ctx.__absentFields?.get(value);
646
+ if (skip) {
647
+ const unknownSkip = [...skip].filter((name) => !expected.has(name));
648
+ if (unknownSkip.length) {
649
+ throw new CodecError(`unexpected absent field(s): ${unknownSkip.join(', ')}`, path);
650
+ }
651
+ let seenSkip = false;
652
+ for (const field of fields) {
653
+ if (skip.has(field.name)) { seenSkip = true; continue; }
654
+ if (seenSkip) {
655
+ throw new CodecError(
656
+ `field ${JSON.stringify(field.name)} follows an absent field — only a trailing run may be omitted`,
657
+ path);
658
+ }
659
+ }
660
+ }
564
661
  const parts = [];
565
662
  for (const field of fields) {
663
+ if (skip?.has(field.name)) continue;
566
664
  if (!(field.name in value)) throw new CodecError(`missing field ${JSON.stringify(field.name)}`, path);
567
665
  parts.push(encodeAt(field.type, value[field.name], ctx, `${path}.${field.name}`, depth + 1));
568
666
  }
@@ -925,6 +1023,43 @@ export function decodeInstructionData(idl, data, { allowTrailing, trailing, abse
925
1023
  throw new CodecError(`unknown absent policy ${JSON.stringify(absent)}`, null);
926
1024
  }
927
1025
  if (!(data instanceof Uint8Array)) throw new CodecError('instruction data must be a Uint8Array/Buffer', null);
1026
+ // The raw little-endian dialect is WIRED at the product boundary (12.11.
1027
+ // PULS Z). Until PULS X the diabloarb interface crashed the codec unnamed
1028
+ // (TypeError over the dict `types`, 102/102 payloads); until this wiring
1029
+ // the same interface was refused by name — honest, but naming a dialect is
1030
+ // not speaking it (PULS Y: the engine decoded 102/102 byte-exactly, and
1031
+ // the card's own next step was this adapter). The branch runs BEFORE the
1032
+ // shape gate: a raw-LE IDL is exactly the shape the dialect gate names as
1033
+ // unsupported, so the engine must answer first. When the bytes defeat the
1034
+ // declared grammar the RawLeError is translated into the codec's error
1035
+ // family — one refusal family at the product boundary, never a RawLeError
1036
+ // leaking past it. The message text stays the engine's own words.
1037
+ if (unsupportedIdlReason(idl) && rawLeDialectReason(idl) === null) {
1038
+ try {
1039
+ const raw = decodeRawLe(idl, data);
1040
+ const { kind, instruction, tag, args, absent, consumed, total, trailing: rawTrailing } = raw;
1041
+ const bytes = data instanceof Uint8Array ? data : Buffer.from(data);
1042
+ return {
1043
+ kind: kind ?? 'instruction',
1044
+ instruction,
1045
+ tag,
1046
+ // The dispatch prefix in hex, like the codec's own `discriminator`
1047
+ // field — here the prefix is the u8 tag the engine dispatched on,
1048
+ // not an Anchor sighash. Consumers that render the prefix (CLI,
1049
+ // src/decode.mjs) therefore see the real wire bytes.
1050
+ discriminator: Buffer.from(data.subarray(0, 1)).toString('hex'),
1051
+ args,
1052
+ absent: absent ?? [],
1053
+ consumed,
1054
+ total,
1055
+ trailing: rawTrailing ?? { count: Math.max(0, bytes.length - consumed), hex: '' },
1056
+ dialect: 'raw-le',
1057
+ };
1058
+ } catch (error) {
1059
+ if (!(error instanceof RawLeError)) throw error;
1060
+ throw new CodecError(error.message, error.path);
1061
+ }
1062
+ }
928
1063
  const ctx = context(idl);
929
1064
  const index = instructionIndex(idl);
930
1065
  const prefixLength = Math.min(8, data.length);
@@ -948,6 +1083,14 @@ export function decodeInstructionData(idl, data, { allowTrailing, trailing, abse
948
1083
  const args = {};
949
1084
  const absentArgs = [];
950
1085
  const declaredArgs = instruction.args ?? [];
1086
+ // Struct-internal absences (the payload ends INSIDE a declared struct
1087
+ // argument, on that struct's own field boundary) surface as a marker the
1088
+ // struct decoder leaves on the decoded value. The struct decoder only
1089
+ // produces that marker under ctx.__absentPolicy === 'report' — every other
1090
+ // entry into the codec leaves these inert.
1091
+ ctx.__absentPolicy = absentPolicy;
1092
+ ctx.__absentCollected = absentArgs;
1093
+ ctx.__absentTrail = [];
951
1094
  for (const [position, argument] of declaredArgs.entries()) {
952
1095
  if (!argument || typeof argument.name !== 'string' || !('type' in argument)) {
953
1096
  throw new CodecError(`instruction ${instruction.name} has a malformed argument`, null);
@@ -962,8 +1105,23 @@ export function decodeInstructionData(idl, data, { allowTrailing, trailing, abse
962
1105
  }
963
1106
  // With absent:'error' (default) this falls through to decodeAt, which
964
1107
  // reports `truncated input` — the behaviour every existing caller pins.
965
- assign(args, argument.name, decodeAt(argument.type, cursor, ctx, `${instruction.name}.${argument.name}`, 0), instruction.name);
1108
+ ctx.__absentTrail.push(argument.name);
1109
+ let value;
1110
+ try {
1111
+ value = decodeAt(argument.type, cursor, ctx, `${instruction.name}.${argument.name}`, 0);
1112
+ } finally {
1113
+ ctx.__absentTrail.pop();
1114
+ }
1115
+ // A struct the wire stopped inside carries the `__absentFields` marker;
1116
+ // peel it off so the decoded args never show codec-internal keys. The
1117
+ // dotted names were already collected into absentArgs. The strip is
1118
+ // recursive: a struct nested in a struct carries its own marker.
1119
+ stripAbsentMarkers(value);
1120
+ assign(args, argument.name, value, instruction.name);
966
1121
  }
1122
+ delete ctx.__absentPolicy;
1123
+ delete ctx.__absentCollected;
1124
+ delete ctx.__absentTrail;
967
1125
  const trailingCount = cursor.remaining;
968
1126
  const trailingHex = trailingCount > 0
969
1127
  ? Buffer.from(data.subarray(cursor.offset)).toString('hex')
@@ -989,6 +1147,26 @@ export function decodeInstructionData(idl, data, { allowTrailing, trailing, abse
989
1147
  };
990
1148
  }
991
1149
 
1150
+ /**
1151
+ * Remove the codec-internal `__absentFields` markers from a decoded value.
1152
+ *
1153
+ * The marker is how a struct tells the instruction level "the wire ended on
1154
+ * my field boundary, and THESE fields are the ones it did not carry". It is
1155
+ * bookkeeping, not data: a consumer must never see it inside the decoded
1156
+ * args. Recursive, because a struct nested in a struct carries its own
1157
+ * marker.
1158
+ */
1159
+ function stripAbsentMarkers(value) {
1160
+ if (!value || typeof value !== 'object') return value;
1161
+ if (Array.isArray(value)) {
1162
+ for (const item of value) stripAbsentMarkers(item);
1163
+ return value;
1164
+ }
1165
+ delete value.__absentFields;
1166
+ for (const key of Object.keys(value)) stripAbsentMarkers(value[key]);
1167
+ return value;
1168
+ }
1169
+
992
1170
  /**
993
1171
  * Encode one instruction payload (discriminator + args) using the IDL.
994
1172
  *
@@ -998,6 +1176,17 @@ export function decodeInstructionData(idl, data, { allowTrailing, trailing, abse
998
1176
  * layout, and byte-exactness is the only evidence this function offers.
999
1177
  */
1000
1178
  export function encodeInstructionData(idl, name, args, { absent } = {}) {
1179
+ // Same dialect branch as decodeInstructionData: a raw-LE interface is
1180
+ // served by its engine, byte for byte, and every refusal comes back in
1181
+ // the codec's own error family.
1182
+ if (unsupportedIdlReason(idl) && rawLeDialectReason(idl) === null) {
1183
+ try {
1184
+ return encodeRawLe(idl, name, args, { absent });
1185
+ } catch (error) {
1186
+ if (!(error instanceof RawLeError)) throw error;
1187
+ throw new CodecError(error.message, error.path);
1188
+ }
1189
+ }
1001
1190
  const ctx = context(idl);
1002
1191
  const instruction = instructionByName(idl, name);
1003
1192
  const bytes = discriminatorBytes(instruction) ?? (() => {
@@ -1009,6 +1198,10 @@ export function encodeInstructionData(idl, name, args, { absent } = {}) {
1009
1198
  const known = new Set(declared.map((argument) => argument.name));
1010
1199
  const absentSet = new Set(Array.isArray(absent) ? absent : []);
1011
1200
  for (const fieldName of absentSet) {
1201
+ // A dotted name names a field INSIDE a declared struct argument; it is
1202
+ // validated against that struct further down, where the struct's own
1203
+ // field list is known.
1204
+ if (fieldName.includes('.')) continue;
1012
1205
  if (!known.has(fieldName)) {
1013
1206
  throw new CodecError(`unexpected argument(s) for ${instruction.name}: ${fieldName}`, instruction.name);
1014
1207
  }
@@ -1030,13 +1223,68 @@ export function encodeInstructionData(idl, name, args, { absent } = {}) {
1030
1223
  if (unknown.length) {
1031
1224
  throw new CodecError(`unexpected argument(s) for ${instruction.name}: ${unknown.join(', ')}`, null);
1032
1225
  }
1033
- const parts = [bytes];
1034
- for (const argument of declared) {
1035
- if (absentSet.has(argument.name)) continue;
1036
- if (!(argument.name in args)) throw new CodecError(`missing argument ${JSON.stringify(argument.name)}`, instruction.name);
1037
- parts.push(encodeAt(argument.type, args[argument.name], ctx, `${instruction.name}.${argument.name}`, 0));
1226
+ // Struct-internal absent fields: an absent name with a dot names a field
1227
+ // INSIDE a declared struct argument (swapParams.builderFeePpm). Resolve
1228
+ // each dotted path to the value object that owns the field, and record the
1229
+ // omission on the encode context so encodeAt's struct branch skips it —
1230
+ // with the same rules the top level enforces: only a trailing run of that
1231
+ // struct's fields may be omitted, and the value must not carry the field.
1232
+ // The encoder is the inverse of the decoder's struct-boundary marker: the
1233
+ // dotted absent list is exactly what the decode reported.
1234
+ const nestedAbsent = new Map(); // value object -> Set(struct-relative field names)
1235
+ for (const fieldName of absentSet) {
1236
+ if (!fieldName.includes('.')) continue;
1237
+ const parts = fieldName.split('.');
1238
+ if (!known.has(parts[0])) {
1239
+ throw new CodecError(`unexpected argument(s) for ${instruction.name}: ${parts[0]}`, instruction.name);
1240
+ }
1241
+ const ownerName = parts[0];
1242
+ const ownerType = declared.find((argument) => argument.name === ownerName)?.type;
1243
+ // Walk the INTERMEDIATE segments through named structs; the LAST segment
1244
+ // is the omitted field itself, validated against the struct it lives in.
1245
+ let containerType = ownerType;
1246
+ let containerValue = args[ownerName];
1247
+ for (const part of parts.slice(1, -1)) {
1248
+ const fields = structFieldsForAbsentPath(containerType, ctx, fieldName);
1249
+ const nextField = fields.find((field) => field.name === part);
1250
+ if (!nextField) {
1251
+ throw new CodecError(`unexpected absent field(s) for ${instruction.name}: ${fieldName}`, instruction.name);
1252
+ }
1253
+ containerType = nextField.type;
1254
+ if (!containerValue || typeof containerValue !== 'object' || !(part in containerValue)) {
1255
+ throw new CodecError(
1256
+ `absent path ${JSON.stringify(fieldName)} crosses a field the value does not carry`, instruction.name);
1257
+ }
1258
+ containerValue = containerValue[part];
1259
+ }
1260
+ const fields = structFieldsForAbsentPath(containerType, ctx, fieldName);
1261
+ const leaf = parts[parts.length - 1];
1262
+ if (!fields.some((field) => field.name === leaf)) {
1263
+ throw new CodecError(`unexpected absent field(s) for ${instruction.name}: ${fieldName}`, instruction.name);
1264
+ }
1265
+ if (!containerValue || typeof containerValue !== 'object' || Array.isArray(containerValue)) {
1266
+ throw new CodecError(`expected a struct object for ${JSON.stringify(fieldName)}, got ${describe(containerValue)}`, instruction.name);
1267
+ }
1268
+ if (leaf in containerValue) {
1269
+ throw new CodecError(`argument ${JSON.stringify(fieldName)} is both absent and present`, instruction.name);
1270
+ }
1271
+ if (!nestedAbsent.has(containerValue)) nestedAbsent.set(containerValue, new Set());
1272
+ nestedAbsent.get(containerValue).add(leaf);
1273
+ }
1274
+ if (nestedAbsent.size) {
1275
+ ctx.__absentFields = nestedAbsent;
1276
+ }
1277
+ try {
1278
+ const parts = [bytes];
1279
+ for (const argument of declared) {
1280
+ if (absentSet.has(argument.name)) continue;
1281
+ if (!(argument.name in args)) throw new CodecError(`missing argument ${JSON.stringify(argument.name)}`, instruction.name);
1282
+ parts.push(encodeAt(argument.type, args[argument.name], ctx, `${instruction.name}.${argument.name}`, 0));
1283
+ }
1284
+ return Buffer.concat(parts);
1285
+ } finally {
1286
+ delete ctx.__absentFields;
1038
1287
  }
1039
- return Buffer.concat(parts);
1040
1288
  }
1041
1289
 
1042
1290
  /**
package/src/dialect.mjs CHANGED
@@ -43,6 +43,7 @@
43
43
  // shank take the same paths as before, byte for byte.
44
44
 
45
45
  import { isShankDiscriminant } from './shank.mjs';
46
+ import { rawLeDialectReason } from './rawle.mjs';
46
47
 
47
48
  const isPlainObject = (value) => !!value && typeof value === 'object' && !Array.isArray(value);
48
49
 
@@ -107,7 +108,12 @@ export function unsupportedIdlReason(idl) {
107
108
  * lists.
108
109
  */
109
110
  export function idlDialect(idl) {
110
- if (unsupportedIdlReason(idl)) return 'unsupported';
111
+ // A raw-LE file is SUPPORTED now (PULS Z wiring) but is not one of the
112
+ // three Anchor-family dialects this classifier enumerates; it reports
113
+ // its own name so callers can tell a wired dialect from a refused one.
114
+ if (unsupportedIdlReason(idl)) {
115
+ return rawLeDialectReason(idl) === null ? 'raw-le' : 'unsupported';
116
+ }
111
117
  const instructions = idl.instructions;
112
118
  if (instructions.length === 0) return 'anchor';
113
119
  for (const instruction of instructions) {
@@ -143,3 +149,32 @@ export function assertIdlShape(idl, makeError) {
143
149
  }
144
150
  throw error;
145
151
  }
152
+
153
+ /**
154
+ * The refusal reason ONLY when no engine in the product can read the file.
155
+ * Since 12.11. PULS Z the raw little-endian engine (src/rawle.mjs) is wired
156
+ * into the codec: an IDL the Anchor codec refuses by SHAPE but the engine
157
+ * claims grammar for is decodable, so the store write gate, the store read
158
+ * gate and the discriminator derivation must not refuse it. Every other
159
+ * unsupported shape keeps the exact named refusal of `unsupportedIdlReason`.
160
+ * The codec's own dispatch (src/codec.mjs) asks the same two questions in
161
+ * the same order: is it outside the Anchor shape, and is it inside the
162
+ * engine grammar?
163
+ */
164
+ export function unreadableDialectReason(idl) {
165
+ const reason = unsupportedIdlReason(idl);
166
+ if (!reason) return null;
167
+ return rawLeDialectReason(idl) === null ? null : reason;
168
+ }
169
+
170
+ /** Like assertIdlShape, but raw-LE-readable files pass — the engine speaks
171
+ * them now (PULS Z). Same named refusal and same error code otherwise. */
172
+ export function assertReadableDialect(idl, makeError) {
173
+ const reason = unreadableDialectReason(idl);
174
+ if (!reason) return;
175
+ const error = makeError(reason);
176
+ if (error && typeof error === 'object') {
177
+ error.code = UNSUPPORTED_DIALECT_CODE;
178
+ }
179
+ throw error;
180
+ }
@@ -28,7 +28,7 @@
28
28
 
29
29
  import { createHash } from 'node:crypto';
30
30
  import { isShankDiscriminant } from './shank.mjs';
31
- import { unsupportedIdlReason } from './dialect.mjs';
31
+ import { unreadableDialectReason, idlDialect } from './dialect.mjs';
32
32
 
33
33
  const sha256 = (text) => createHash('sha256').update(text).digest();
34
34
 
@@ -119,14 +119,29 @@ export function verifyDerivedDiscriminators(idl) {
119
119
  * @returns {{ idl: object, filled: { instructions: number, events: number }, conflicts: object[] }}
120
120
  */
121
121
  export function withDerivedDiscriminators(idl) {
122
- // A dialect the codec cannot read gets NO derived table at all. Measured
123
- // 2026-11-12 on the diabloarb interface: the derivation filled all four
124
- // tag-dispatch instructions with Anchor sighashes that match no wire — a
125
- // table that looks authoritative and decodes nothing. The refusal is named
126
- // (see src/dialect.mjs) and identical to the codec's own.
127
- const refusal = unsupportedIdlReason(idl);
122
+ // A dialect NO engine in the product can read gets NO derived table at
123
+ // all. Measured 2026-11-12 on the diabloarb interface: the derivation
124
+ // filled all four tag-dispatch instructions with Anchor sighashes that
125
+ // match no wire — a table that looks authoritative and decodes nothing.
126
+ // The refusal is named (see src/dialect.mjs) and identical to the
127
+ // codec's own. SINCE THE PULS-Z WIRING the raw little-endian engine is
128
+ // a product engine: a raw-LE interface carries its own tag dispatch (a
129
+ // u8 `tag`, not a sighash) and passes this gate to the codec, which
130
+ // serves it from src/rawle.mjs — no Anchor table is derived for it and
131
+ // none is needed.
132
+ const refusal = unreadableDialectReason(idl);
128
133
  if (refusal) throw new Error(refusal);
129
134
  const copy = structuredClone(idl);
135
+ // A raw-LE file (PULS Z wiring) dispatches on the u8 tag its own file
136
+ // declares — Anchor sighash derivation for it is the measured PULS X
137
+ // defect one gate change later: filled {instructions: 4, events: 0}, a
138
+ // table that looks authoritative and matches no wire byte. The honest
139
+ // table for this dialect is NO derived table, and the codec serves the
140
+ // file from src/rawle.mjs. The file still passes the gate (an engine
141
+ // reads it); only the Anchor filling is withheld.
142
+ if (idlDialect(idl) === 'raw-le') {
143
+ return { idl: copy, filled: { instructions: 0, events: 0 }, conflicts: [] };
144
+ }
130
145
  let instructions = 0;
131
146
  let events = 0;
132
147
  const conflicts = [];
package/src/rawle.mjs CHANGED
@@ -42,15 +42,19 @@
42
42
  // A remainder that is neither exactly the option size nor zero is a
43
43
  // named refusal (trailing bytes the layout does not explain).
44
44
  //
45
- // NOT WIRED INTO THE PRODUCT. No call site in codec/union/adopt/store-read
46
- // or the CLI routes through this engine yet — PULS X's named refusal stays
47
- // the product's answer at every boundary, byte for byte, and the dialect
48
- // gate is untouched. That is deliberate: making support the product answer
49
- // changes 19 pinned faces of the PULS X measurement (its artifacts and
50
- // tools assert the refusal of THIS interface), and the release it belongs
51
- // to must state the change. The engine + measurement first, the wiring
52
- // with the pins it rewrites — this is the same order every pulse in this
53
- // repo has worked in.
45
+ // WIRED INTO THE PRODUCT SINCE PULS Z (2026-11-12): src/codec.mjs routes
46
+ // every decode/encode of a file OUTSIDE the Anchor shape but INSIDE this
47
+ // grammar through this engine — instruction decode/encode, the union
48
+ // probe, store adopt/read, and the discriminator derivation gate (an
49
+ // engine-readable file passes it; no Anchor table is derived for it).
50
+ // RawLeError is translated into CodecError at the boundary, so the
51
+ // product keeps ONE refusal family. Everything else — every anchor/shank/
52
+ // name-only file — takes the exact same codec paths as before, byte for
53
+ // byte, and files no engine can read keep PULS X's named refusal.
54
+ // The face this wiring turns: diabloarb's 102 sample payloads, refused
55
+ // by name until now, decode byte-exactly through the product entry. The
56
+ // pins it bends are the PULS X refusal faces for THIS interface, and the
57
+ // measurement that proves them is tools/measure-rawle-product.mjs.
54
58
  //
55
59
  // Byte-exactness note: decoded integers come back as Number (<= 4 bytes)
56
60
  // or BigInt (8 bytes), exactly like src/codec.mjs's own convention, and
@@ -114,23 +114,51 @@ const VERIFIED_SHORT_FORMS = [
114
114
  absent: ['is_cashback_enabled', 'creator_fee_bps', 'is_holder_reward'],
115
115
  witnesses: 1,
116
116
  },
117
+ // --- 2026-11-12: the first short form whose missing field sits INSIDE a
118
+ // struct argument (research/archer-legacy-swap-witness-2026-11-12.json).
119
+ // Every form above ends at a TOP-LEVEL argument boundary; this one ends on
120
+ // the boundary of the LAST field inside `swapParams`, so the absent name
121
+ // is dotted (`swapParams.builderFeePpm`) and the codec's struct-boundary
122
+ // rule is what makes it decodable. Evidence: a Swap executed at slot
123
+ // 468261978 with err:null whose instruction bytes are byte-identical to
124
+ // the 19-byte payload (checked against the RPC witness twice: `json`
125
+ // encoding base58 -> 0f430000000000000000000000000000000100), the Archer
126
+ // CPI's own `success` line, and the pinned IDL's SwapParams docs which
127
+ // describe the shorter legacy payload as a supported form read as a 0 fee.
128
+ {
129
+ program: 'Archer8kgiavM61GyusMzaaS2ft5sALtNsD1HxkUPMhy',
130
+ discriminator: '0f',
131
+ bytes: 19,
132
+ instruction: 'Swap',
133
+ absent: ['swapParams.builderFeePpm'],
134
+ witnesses: 1,
135
+ missingFieldInsideStruct: true,
136
+ },
117
137
  ];
118
138
 
119
139
  export const SHORT_FORM_SOURCE =
120
140
  'research/short-form-wire-forms-2026-11-03.json + research/short-form-wire-forms-2026-11-04.json '
121
- + '(chain witnesses: err:null, byte-identical payload, matching program log line)';
141
+ + '(chain witnesses: err:null, byte-identical payload, matching program log line) '
142
+ + 'research/archer-legacy-swap-witness-2026-11-12.json (chain witness: err:null, byte-identical payload)';
122
143
 
123
144
  export const VERIFIED = VERIFIED_SHORT_FORMS;
124
145
  export { VERIFIED_SHORT_FORMS };
125
146
 
126
147
  /** Look up a payload in the chain-verified short-form table; null when absent. */
127
148
  export function verifiedShortForm(program, data) {
149
+ // Struct-internal short forms may carry a discriminator SHORTER than 8
150
+ // bytes (measured: Archer's shank tag, 1 byte). The old `data.length < 8`
151
+ // guard refused any lookup that could not yield such a match: a 19-byte
152
+ // payload with a 1-byte tag reaches the table fine once the prefix is
153
+ // clipped to min(8, len) — exactly the rule the codec's instruction index
154
+ // already uses for short stored discriminators.
128
155
  if (!program || !(data instanceof Uint8Array)) return null;
129
- if (data.length < 8) return null;
130
- const discriminator = Buffer.from(data.subarray(0, 8)).toString('hex');
156
+ if (data.length === 0) return null;
157
+ const prefixLength = Math.min(8, data.length);
158
+ const discriminator = Buffer.from(data.subarray(0, prefixLength)).toString('hex');
131
159
  return VERIFIED_SHORT_FORMS.find((entry) =>
132
160
  entry.program === program
133
- && entry.discriminator === discriminator
161
+ && discriminator.startsWith(entry.discriminator)
134
162
  && entry.bytes === data.length) ?? null;
135
163
  }
136
164
 
@@ -33,7 +33,7 @@ import { canonicalJson } from './canonical.mjs';
33
33
  import { compareReceiptNames } from './receipt-order.mjs';
34
34
  import { classifyReceiptFiles, isReceiptFile } from './receipt-name.mjs';
35
35
  import { receiptIntegrity } from './receipt-digest.mjs';
36
- import { assertIdlShape, declaredProgram } from './dialect.mjs';
36
+ import { assertIdlShape, assertReadableDialect, declaredProgram } from './dialect.mjs';
37
37
 
38
38
  export { canonicalJson } from './canonical.mjs';
39
39
 
@@ -80,11 +80,14 @@ export async function readVerified(storeDir, program, receiptName) {
80
80
  }
81
81
  if (!Array.isArray(idl.instructions)) throw Error(`stored object for ${program} has no instructions`);
82
82
  // Same two checks as the write path (src/adopt.mjs), applied on every
83
- // read: a dialect the codec cannot read is refused by NAME before any
84
- // decode can crash unnamed, and a declared program that differs from the
85
- // receipt's program is a re-labelling no matter which field carries the
86
- // declaration (`address`, `metadata.address` or `program_id`).
87
- assertIdlShape(idl, (reason) => {
83
+ // read: a dialect NO engine of the product can read is refused by NAME
84
+ // before any decode can crash unnamed, and a declared program that
85
+ // differs from the receipt's program is a re-labelling no matter which
86
+ // field carries the declaration (`address`, `metadata.address` or
87
+ // `program_id`). Since the PULS-Z wiring a stored raw-LE object reads
88
+ // through the engine (src/rawle.mjs) — a store that refused it by name
89
+ // now decodes instead, which is the point of the wiring.
90
+ assertReadableDialect(idl, (reason) => {
88
91
  const error = Error(`stored object ${fingerprint} for ${program}: ${reason}`);
89
92
  error.code = 'unsupported-idl-dialect';
90
93
  return error;