sememe 0.4.21 → 0.4.22

package/README.md

@@ -132,7 +132,7 @@ sememe is a local, content-addressed schema store with a verification contract:
 ## Install (as a consumer, not a contributor)
 
 ```bash
-npm install github:auroraxo/sememe#v0.4.21   # tagged release; no npm account needed
+npm install github:auroraxo/sememe#v0.4.22   # tagged release; no npm account needed
 ```
 
 ```js
@@ -151,7 +151,7 @@ The **library** refuses the same way: importing `sememe/pull` or `sememe/gap` wi
 **From an install (not a clone), the CLI is `npx sememe …`** — npm exec resolves the package's own `bin` from `node_modules`; the bare command `sememe` is not on your `PATH` after a plain `npm install`, and `node bin/sememe.mjs` works only inside a clone of the repository (both measured 2026-11-05 from a real tarball install). The consumer journey from the install, verbatim:
 
 ```bash
-npm install github:auroraxo/sememe#v0.4.21 @solana/web3.js   # acquisition needs the peer
+npm install github:auroraxo/sememe#v0.4.22 @solana/web3.js   # acquisition needs the peer
 npx sememe pull whirLbMiicVdio4qvUfM5KAg6Ct8VwpYzGff3uctyCc --store ./store
 npx sememe decode whirLbMiicVdio4qvUfM5KAg6Ct8VwpYzGff3uctyCc 2b04ed0b1ac91e62a40d0100000000000100000000000000af331ba8327fbb35b1c4feff00000000010001010000000601 --store ./store
 ```
@@ -2570,6 +2570,116 @@ way shank was. What this fix guarantees is that until that happens, the
 interface is refused by name at every boundary and never crashes the codec
 or pretends to be decodable.
 
+## The product SPEAKS the dialect it named: the raw-LE engine is wired (2026-11-12)
+
+PULS X named the refusal ("naming a dialect is not speaking it") and PULS Y
+built the speaker — an engine for exactly the grammar the diabloarb file
+DECLARES (`encoding: "little-endian; u8/u16/u32/u64 unsigned, i64 signed; no
+padding; no length prefixes except the u8 counters named in each header"`):
+u8-tag dispatch, dict `types` with declared fixed sizes, counter-prefixed
+vectors (`Base[n_bases]`, `u8[n_hops]`), one byte-decided optional trailing
+field. 102/102 committed payloads decoded byte-exactly, and the product
+boundary stayed untouched — deliberately: capability first, the wiring with
+the pins it rewrites.
+
+**This pulse is the wiring** (the card's own declared next step):
+
+- `src/codec.mjs` routes every decode/encode of a file OUTSIDE the Anchor
+  shape but INSIDE the engine grammar through `src/rawle.mjs`, before the
+  shape gate. The result carries the codec's own fields (`discriminator` is
+  the dispatch prefix in hex; `dialect: 'raw-le'`), and every engine refusal
+  is translated into `CodecError` — one refusal family at the product
+  boundary, never a `RawLeError` leak, never a crash.
+- `src/dialect.mjs` splits the question "can anything read this file?" from
+  "can the Anchor codec read it?": `unreadableDialectReason`/`assertReadableDialect`
+  pass an engine-readable file and keep the exact named refusal (code
+  `unsupported-idl-dialect`) for everything else; `idlDialect` reports
+  `'raw-le'` for a wired dialect.
+- The store gates (`src/adopt.mjs` write path, `src/store-read.mjs` read
+  path) use the readable-dialect guard: a raw-LE object adopts under its
+  own program id, reads back, and the CLI loop `sememe adopt` → `sememe
+  decode` works end-to-end offline (measured 2/2). Mislabelling under a
+  foreign program id stays refused via `program_id` — the wiring opens the
+  dialect gate, not the re-labelling gate.
+- The derivation (`src/discriminators.mjs`) passes the file WITHOUT filling
+  any Anchor sighash table (the PULS X fill defect stays dead — measured
+  filled {instructions: 0, events: 0}); the union (`src/union.mjs`)
+  decodes through the engine like through any source.
+
+Measured two-tree, byte-identical probe
+([`research/rawle-product-2026-11-12.json`](research/rawle-product-2026-11-12.json),
+pinned in `test/rawle-product-measurement.test.mjs` with a full
+gh-gated re-derivation): prefix tree = the PULS Y engine commit `05e6643`
+(engine present, product refuses — 0/102, store loop closed), this tree =
+102/102 decoded AND 102/102 byte-exact re-encoded at `decodeInstructionData`,
+5/5 product-level refusals as named `CodecError` faces (unknown tag,
+truncated payload, trailing bytes, unknown instruction on encode, missing
+argument on encode), derivation passes with NO table, adopt/read/union open,
+CLI store loop 2/2, Whirlpool control unchanged, a file that defeats BOTH
+engines keeps the exact PULS X named refusal at codec AND union. 24
+falsification counts, all zero. The PULS X measurement
+([`research/dialect-named-refusal-2026-11-12.json`](research/dialect-named-refusal-2026-11-12.json))
+was re-derived in its invariant form: its delta now reads crash → decodes
+through the engine, fill {4,0} → {0,0} — the no-crash, no-fill, no-mislabel
+invariants pinned on every payload. The PULS Y artifact stays the frozen
+pre-wiring baseline; its tool now exits 3 NO VERDICT in a wired tree BY
+DESIGN (stated in the tool header).
+
+**Tests: 1 new pin file** (`test/rawle-product-measurement.test.mjs`, 12
+tests) plus the bent faces in `test/dialect-named-refusal.test.mjs` (D1b),
+`test/rawle-decoder.test.mjs` (G1) and
+`test/dialect-named-refusal-measurement.test.mjs`. Suite 912 → **933: 929
+pass / 4 named network skips / 0 fail**. Product code changed; the pin is
+the next release.
+
+## The wiring SHIPS in v0.4.22 — and the ecosystem number it moves is +102 for the user who adopts, not the shipped store (2026-11-12)
+
+This release carries the raw-LE wiring above to the install channel: the
+pinned tag now contains `src/rawle.mjs` and every wired boundary, so
+`npm install github:auroraxo/sememe#v0.4.22` degrades nobody (Whirlpool,
+shank, name-only anchor controls pin unchanged) and answers the diabloarb
+dialect where v0.4.21 refused it by name.
+
+The wiring also prompted the card's declared coverage re-measurement, and
+the honest answer is TWO-HALVED (`tools/measure-diablo-adoption-coverage.mjs`
+→ `research/diablo-adoption-coverage-2026-11-12.json`, pinned by
+`test/diablo-adoption-coverage.test.mjs`, 9 tests, gh-gated re-derivation
+included):
+
+- **At the shipped store the headline does NOT move.** Re-running the
+  committed 2026-11-11 fixture through the coverage tool on this tree
+  reproduces the committed artifact exactly: **88.5%, DiabLokxis 0 decoded
+  / 102 no-interface.** The wiring changed the codec, not the store, and
+  the diabloarb copy cannot ship inside the store: `cryptodiablo/diabloarb`
+  carries **no license** at the pin `f5005ac4` — live re-checked inside the
+  pin test (GitHub license API 404 at the pin AND at HEAD, repo field
+  null), not narrated. The bytes stay out of the repository and the
+  package; `committedStoreModified: false` (readdir + per-file hash digest
+  before/after).
+- **For the user who adopts, the 102 are real.** ONE `adopt` of the
+  hash-gated first-party copy (canonical fingerprint
+  `d6d4e8aa…` verified before anything runs) through the product's own
+  write path, into a throwaway copy of the committed store, and the SAME
+  report tool answers **102/102 decoded, 0 refused, 102 byte-exact**:
+  headline **88.5% → 89.0%** of measured traffic, outside the watchlist
+  **62.4% → 64.4%**. Every one of the 102 rides the **adopted** evidence
+  column (`decodedVia.adopted +102`; `pulled` moves by exactly zero) — the
+  same class split the pfeeU, manifest and archer measurements keep, so an
+  adoption can never silently inflate the on-chain-pull number.
+
+So "102 jetzt zählbar" was half true: countable **yes** — as adoption-path
+coverage, with a name and a byte-exact check — and **not** as a shipped
+headline. The acquisition gap at that program is a LICENSE gap, not a
+decoder gap: the one thing between the user and those 102 payloads is a
+licensed first-party copy, which is exactly what `sememe adopt` exists to
+carry while origin stays declared.
+
+Suite after this release: **954 tests / 950 pass / 4 named network skips /
+0 fail** (945 → 954: the 9 new pin tests). CI green on `main`
+(43287274405) and on the published tag v0.4.22 (43287274209); the release
+surface re-measured against that tag is clean (newest = latest = pinned,
+0 defects — `research/release-surface-2026-11-12b.json`).
+
 ## The remaining 35 publish-nothing programs, classified against foreign sources (2026-11-04)
 
 With Lighthouse decoded under shank, exactly **35 programs (87 payloads)** remained

package/package.json

@@ -1,6 +1,6 @@
 {
   "name": "sememe",
-  "version": "0.4.21",
+  "version": "0.4.22",
   "description": "Local, fingerprint-verified Solana program interfaces: pull an IDL once, then decode instructions, accounts and events offline. Zero dependencies on the decode path. Ships derived wire tables for System, ComputeBudget, SPL Token, Token-2022 (incl. its token-metadata fallback), Raydium AMM v4 and the SPL Associated Token Account, pinned to upstream Rust.",
   "license": "MIT",
   "type": "module",

package/src/adopt.mjs

@@ -39,7 +39,7 @@ import path from 'node:path';
 import { canonicalJson } from './canonical.mjs';
 import { writeOnce } from './store-write.mjs';
 import { originDigestOf, receiptDigestOf } from './receipt-digest.mjs';
-import { assertIdlShape, declaredProgram } from './dialect.mjs';
+import { assertIdlShape, assertReadableDialect, declaredProgram } from './dialect.mjs';
 
 const sha256 = (data) => createHash('sha256').update(data).digest('hex');
 
@@ -117,12 +117,13 @@ export async function adopt(programInput, filePath, storeDir, { source = 'publis
     }
     throw Error(`${filePath} declares program ${declared}, not ${program} — refusing to re-label a foreign interface`);
   }
-  // Gate 2 — the dialect gate runs at the store's WRITE path too: an
-  // interface the codec cannot read (measured 2026-11-12: diabloarb's
-  // tag/data dispatch with dict `types`) is refused HERE, not after
-  // adoption when every decode crashes unnamed. Same reason text as the
-  // codec's guard (src/dialect.mjs).
-  assertIdlShape(idl, (reason) => Error(`${filePath}: ${reason}`));
+  // Gate 2 — the dialect gate at the store's WRITE path: a dialect NO
+  // engine of the product can read is refused HERE by name, before
+  // adoption can store an undecodable interface. SINCE THE PULS-Z WIRING
+  // a raw-LE file passes (the codec serves it from src/rawle.mjs),
+  // everything else keeps the named refusal (measured 2026-11-12:
+  // diabloarb's dict `types` was the face of that refusal).
+  assertReadableDialect(idl, (reason) => Error(`${filePath}: ${reason}`));
 
   const json = Buffer.from(canonicalJson(idl) + '\n');
   const fingerprint = sha256(json);

package/src/codec.mjs

@@ -20,7 +20,8 @@
 
 import { encodeBase58, decodeBase58 } from './base58.mjs';
 import { isShankIdl, isShankDiscriminant, shankDiscriminatorBytes, SHANK_LEB128_TYPES } from './shank.mjs';
-import { assertIdlShape } from './dialect.mjs';
+import { assertIdlShape, unsupportedIdlReason } from './dialect.mjs';
+import { rawLeDialectReason, decodeRawLe, encodeRawLe, RawLeError } from './rawle.mjs';
 
 const MAX_DEPTH = 32;
 // Instruction data is bounded by transaction size; a hostile length prefix
@@ -925,6 +926,43 @@ export function decodeInstructionData(idl, data, { allowTrailing, trailing, abse
     throw new CodecError(`unknown absent policy ${JSON.stringify(absent)}`, null);
   }
   if (!(data instanceof Uint8Array)) throw new CodecError('instruction data must be a Uint8Array/Buffer', null);
+  // The raw little-endian dialect is WIRED at the product boundary (12.11.
+  // PULS Z). Until PULS X the diabloarb interface crashed the codec unnamed
+  // (TypeError over the dict `types`, 102/102 payloads); until this wiring
+  // the same interface was refused by name — honest, but naming a dialect is
+  // not speaking it (PULS Y: the engine decoded 102/102 byte-exactly, and
+  // the card's own next step was this adapter). The branch runs BEFORE the
+  // shape gate: a raw-LE IDL is exactly the shape the dialect gate names as
+  // unsupported, so the engine must answer first. When the bytes defeat the
+  // declared grammar the RawLeError is translated into the codec's error
+  // family — one refusal family at the product boundary, never a RawLeError
+  // leaking past it. The message text stays the engine's own words.
+  if (unsupportedIdlReason(idl) && rawLeDialectReason(idl) === null) {
+    try {
+      const raw = decodeRawLe(idl, data);
+      const { kind, instruction, tag, args, absent, consumed, total, trailing: rawTrailing } = raw;
+      const bytes = data instanceof Uint8Array ? data : Buffer.from(data);
+      return {
+        kind: kind ?? 'instruction',
+        instruction,
+        tag,
+        // The dispatch prefix in hex, like the codec's own `discriminator`
+        // field — here the prefix is the u8 tag the engine dispatched on,
+        // not an Anchor sighash. Consumers that render the prefix (CLI,
+        // src/decode.mjs) therefore see the real wire bytes.
+        discriminator: Buffer.from(data.subarray(0, 1)).toString('hex'),
+        args,
+        absent: absent ?? [],
+        consumed,
+        total,
+        trailing: rawTrailing ?? { count: Math.max(0, bytes.length - consumed), hex: '' },
+        dialect: 'raw-le',
+      };
+    } catch (error) {
+      if (!(error instanceof RawLeError)) throw error;
+      throw new CodecError(error.message, error.path);
+    }
+  }
   const ctx = context(idl);
   const index = instructionIndex(idl);
   const prefixLength = Math.min(8, data.length);
@@ -998,6 +1036,17 @@ export function decodeInstructionData(idl, data, { allowTrailing, trailing, abse
  * layout, and byte-exactness is the only evidence this function offers.
  */
 export function encodeInstructionData(idl, name, args, { absent } = {}) {
+  // Same dialect branch as decodeInstructionData: a raw-LE interface is
+  // served by its engine, byte for byte, and every refusal comes back in
+  // the codec's own error family.
+  if (unsupportedIdlReason(idl) && rawLeDialectReason(idl) === null) {
+    try {
+      return encodeRawLe(idl, name, args, { absent });
+    } catch (error) {
+      if (!(error instanceof RawLeError)) throw error;
+      throw new CodecError(error.message, error.path);
+    }
+  }
   const ctx = context(idl);
   const instruction = instructionByName(idl, name);
   const bytes = discriminatorBytes(instruction) ?? (() => {

package/src/dialect.mjs

@@ -43,6 +43,7 @@
 // shank take the same paths as before, byte for byte.
 
 import { isShankDiscriminant } from './shank.mjs';
+import { rawLeDialectReason } from './rawle.mjs';
 
 const isPlainObject = (value) => !!value && typeof value === 'object' && !Array.isArray(value);
 
@@ -107,7 +108,12 @@ export function unsupportedIdlReason(idl) {
  * lists.
  */
 export function idlDialect(idl) {
-  if (unsupportedIdlReason(idl)) return 'unsupported';
+  // A raw-LE file is SUPPORTED now (PULS Z wiring) but is not one of the
+  // three Anchor-family dialects this classifier enumerates; it reports
+  // its own name so callers can tell a wired dialect from a refused one.
+  if (unsupportedIdlReason(idl)) {
+    return rawLeDialectReason(idl) === null ? 'raw-le' : 'unsupported';
+  }
   const instructions = idl.instructions;
   if (instructions.length === 0) return 'anchor';
   for (const instruction of instructions) {
@@ -143,3 +149,32 @@ export function assertIdlShape(idl, makeError) {
   }
   throw error;
 }
+
+/**
+ * The refusal reason ONLY when no engine in the product can read the file.
+ * Since 12.11. PULS Z the raw little-endian engine (src/rawle.mjs) is wired
+ * into the codec: an IDL the Anchor codec refuses by SHAPE but the engine
+ * claims grammar for is decodable, so the store write gate, the store read
+ * gate and the discriminator derivation must not refuse it. Every other
+ * unsupported shape keeps the exact named refusal of `unsupportedIdlReason`.
+ * The codec's own dispatch (src/codec.mjs) asks the same two questions in
+ * the same order: is it outside the Anchor shape, and is it inside the
+ * engine grammar?
+ */
+export function unreadableDialectReason(idl) {
+  const reason = unsupportedIdlReason(idl);
+  if (!reason) return null;
+  return rawLeDialectReason(idl) === null ? null : reason;
+}
+
+/** Like assertIdlShape, but raw-LE-readable files pass — the engine speaks
+ * them now (PULS Z). Same named refusal and same error code otherwise. */
+export function assertReadableDialect(idl, makeError) {
+  const reason = unreadableDialectReason(idl);
+  if (!reason) return;
+  const error = makeError(reason);
+  if (error && typeof error === 'object') {
+    error.code = UNSUPPORTED_DIALECT_CODE;
+  }
+  throw error;
+}

package/src/discriminators.mjs

@@ -28,7 +28,7 @@
 
 import { createHash } from 'node:crypto';
 import { isShankDiscriminant } from './shank.mjs';
-import { unsupportedIdlReason } from './dialect.mjs';
+import { unreadableDialectReason, idlDialect } from './dialect.mjs';
 
 const sha256 = (text) => createHash('sha256').update(text).digest();
 
@@ -119,14 +119,29 @@ export function verifyDerivedDiscriminators(idl) {
  * @returns {{ idl: object, filled: { instructions: number, events: number }, conflicts: object[] }}
  */
 export function withDerivedDiscriminators(idl) {
-  // A dialect the codec cannot read gets NO derived table at all. Measured
-  // 2026-11-12 on the diabloarb interface: the derivation filled all four
-  // tag-dispatch instructions with Anchor sighashes that match no wire — a
-  // table that looks authoritative and decodes nothing. The refusal is named
-  // (see src/dialect.mjs) and identical to the codec's own.
-  const refusal = unsupportedIdlReason(idl);
+  // A dialect NO engine in the product can read gets NO derived table at
+  // all. Measured 2026-11-12 on the diabloarb interface: the derivation
+  // filled all four tag-dispatch instructions with Anchor sighashes that
+  // match no wire — a table that looks authoritative and decodes nothing.
+  // The refusal is named (see src/dialect.mjs) and identical to the
+  // codec's own. SINCE THE PULS-Z WIRING the raw little-endian engine is
+  // a product engine: a raw-LE interface carries its own tag dispatch (a
+  // u8 `tag`, not a sighash) and passes this gate to the codec, which
+  // serves it from src/rawle.mjs — no Anchor table is derived for it and
+  // none is needed.
+  const refusal = unreadableDialectReason(idl);
   if (refusal) throw new Error(refusal);
   const copy = structuredClone(idl);
+  // A raw-LE file (PULS Z wiring) dispatches on the u8 tag its own file
+  // declares — Anchor sighash derivation for it is the measured PULS X
+  // defect one gate change later: filled {instructions: 4, events: 0}, a
+  // table that looks authoritative and matches no wire byte. The honest
+  // table for this dialect is NO derived table, and the codec serves the
+  // file from src/rawle.mjs. The file still passes the gate (an engine
+  // reads it); only the Anchor filling is withheld.
+  if (idlDialect(idl) === 'raw-le') {
+    return { idl: copy, filled: { instructions: 0, events: 0 }, conflicts: [] };
+  }
   let instructions = 0;
   let events = 0;
   const conflicts = [];

package/src/rawle.mjs

@@ -42,15 +42,19 @@
 //     A remainder that is neither exactly the option size nor zero is a
 //     named refusal (trailing bytes the layout does not explain).
 //
-// NOT WIRED INTO THE PRODUCT. No call site in codec/union/adopt/store-read
-// or the CLI routes through this engine yet — PULS X's named refusal stays
-// the product's answer at every boundary, byte for byte, and the dialect
-// gate is untouched. That is deliberate: making support the product answer
-// changes 19 pinned faces of the PULS X measurement (its artifacts and
-// tools assert the refusal of THIS interface), and the release it belongs
-// to must state the change. The engine + measurement first, the wiring
-// with the pins it rewrites — this is the same order every pulse in this
-// repo has worked in.
+// WIRED INTO THE PRODUCT SINCE PULS Z (2026-11-12): src/codec.mjs routes
+// every decode/encode of a file OUTSIDE the Anchor shape but INSIDE this
+// grammar through this engine — instruction decode/encode, the union
+// probe, store adopt/read, and the discriminator derivation gate (an
+// engine-readable file passes it; no Anchor table is derived for it).
+// RawLeError is translated into CodecError at the boundary, so the
+// product keeps ONE refusal family. Everything else — every anchor/shank/
+// name-only file — takes the exact same codec paths as before, byte for
+// byte, and files no engine can read keep PULS X's named refusal.
+// The face this wiring turns: diabloarb's 102 sample payloads, refused
+// by name until now, decode byte-exactly through the product entry. The
+// pins it bends are the PULS X refusal faces for THIS interface, and the
+// measurement that proves them is tools/measure-rawle-product.mjs.
 //
 // Byte-exactness note: decoded integers come back as Number (<= 4 bytes)
 // or BigInt (8 bytes), exactly like src/codec.mjs's own convention, and

package/src/store-read.mjs

@@ -33,7 +33,7 @@ import { canonicalJson } from './canonical.mjs';
 import { compareReceiptNames } from './receipt-order.mjs';
 import { classifyReceiptFiles, isReceiptFile } from './receipt-name.mjs';
 import { receiptIntegrity } from './receipt-digest.mjs';
-import { assertIdlShape, declaredProgram } from './dialect.mjs';
+import { assertIdlShape, assertReadableDialect, declaredProgram } from './dialect.mjs';
 
 export { canonicalJson } from './canonical.mjs';
 
@@ -80,11 +80,14 @@ export async function readVerified(storeDir, program, receiptName) {
   }
   if (!Array.isArray(idl.instructions)) throw Error(`stored object for ${program} has no instructions`);
   // Same two checks as the write path (src/adopt.mjs), applied on every
-  // read: a dialect the codec cannot read is refused by NAME before any
-  // decode can crash unnamed, and a declared program that differs from the
-  // receipt's program is a re-labelling no matter which field carries the
-  // declaration (`address`, `metadata.address` or `program_id`).
-  assertIdlShape(idl, (reason) => {
+  // read: a dialect NO engine of the product can read is refused by NAME
+  // before any decode can crash unnamed, and a declared program that
+  // differs from the receipt's program is a re-labelling no matter which
+  // field carries the declaration (`address`, `metadata.address` or
+  // `program_id`). Since the PULS-Z wiring a stored raw-LE object reads
+  // through the engine (src/rawle.mjs) — a store that refused it by name
+  // now decodes instead, which is the point of the wiring.
+  assertReadableDialect(idl, (reason) => {
     const error = Error(`stored object ${fingerprint} for ${program}: ${reason}`);
     error.code = 'unsupported-idl-dialect';
     return error;