@pioneer-platform/pioneer-discovery 10.0.8 → 10.2.0

package/.turbo/turbo-build.log

@@ -1,5 +1,4 @@
-
-
-> @pioneer-platform/pioneer-discovery@10.0.8 build /Users/highlander/WebstormProjects/keepkey-stack/projects/pioneer/modules/pioneer/pioneer-discovery
-> tsc -p . && cp src/*.json lib/ 2>/dev/null || true
-
+
+> @pioneer-platform/pioneer-discovery@10.1.0 build /Users/highlander/WebstormProjects/keepkey-stack/projects/pioneer/modules/pioneer/pioneer-discovery
+> tsc -p . && cp src/*.json lib/ 2>/dev/null || true && mkdir -p lib/descriptors && cp src/descriptors/*.json lib/descriptors/ 2>/dev/null || true && mkdir -p lib/signatures && cp src/signatures/*.json lib/signatures/ 2>/dev/null || true
+

package/CHANGELOG.md

@@ -1,5 +1,15 @@
 # @pioneer-platform/pioneer-discovery
 
+## 10.0.9
+
+### Patch Changes
+
+- 5df7310: Add Solana program registry (`src/solana-programs.json`) with 23 canonical programs: System, SPL Token, SPL Token 2022, Associated Token Account, Memo v1/v2, Compute Budget, Stake, Vote, Address Lookup Table, Jupiter Aggregator v6, Raydium AMM v4 / CPMM / CLMM, Orca Whirlpool, Phoenix v1, Metaplex Token Metadata / Core / Auction House, Magic Eden v2, Tensor Swap, Marinade, Drift v2.
+
+  Each entry declares how to extract its instruction discriminator (Anchor 8-byte sighash, SPL 1-byte tag, System 4-byte LE u32, or `none`). Full instruction schemas are included for System transfer/createAccount/allocate, SPL Token transfer/transferChecked/approve/mintTo/burn/close, Token-2022 mirrors, ATA create/createIdempotent/recoverNested, and Compute Budget. Anchor-based DeFi and NFT programs declare name + category only for now; discriminators to be added in follow-ups alongside test fixtures.
+
+  Consumers: KeepKey Vault's clear-signing review UI reads this registry to render human-readable instruction summaries for Solana transactions; firmware-side Insight metadata binding will reuse the same id → program mapping once on-device v0 support lands.
+
 ## 10.0.8
 
 ### Patch Changes

package/ENDPOINTS.md

@@ -0,0 +1,350 @@
+# Pioneer Discovery API — Endpoint Reference
+
+Base URL: `http://localhost:9001` (local) or production deployment
+
+## Quick Start
+
+```bash
+cd modules/pioneer/pioneer-discovery
+
+# Serve locally (zero dependencies, Node.js only)
+pnpm run serve
+
+# Or with custom port
+node scripts/serve.mjs --port 8080
+```
+
+## Data Pipeline
+
+```
+ledger-asset-dapps/  ──┐
+                       ├── import-ledger-descriptors.js ──→ evm-descriptors.json (800 contracts, 2957 selectors)
+THORChain/Maya routers ┘
+4byte.directory standards ──→ import-4byte-selectors.js ──→ universal-selectors.json (462 selectors)
+manual curation           ──→                            ──→ cosmos-msgs.json (28 msg types)
+
+               sign-all.mjs (INSIGHT_MNEMONIC)
+                    │
+     ┌──────────────┼──────────────┐
+     ▼              ▼              ▼
+evm-signed-    token-         cosmos-
+blobs.json     signatures.json  signatures.json
+(2933 blobs)   (8730 sigs)    (28 sigs)
+```
+
+**Re-sign after data changes:**
+```bash
+INSIGHT_MNEMONIC="..." node scripts/sign-all.mjs   # ~82 seconds, 11,691 signatures
+```
+
+---
+
+## Endpoints
+
+### Health & Meta
+
+| Method | Path | Description |
+|--------|------|-------------|
+| GET | `/api/v2/health` | Service health + aggregate stats |
+| GET | `/api/v2/manifest` | Signing verification manifest (public key, formats) |
+| GET | `/api/v2/descriptors/stats` | Detailed registry statistics |
+
+**`GET /api/v2/health`**
+```json
+{
+  "online": true,
+  "service": "pioneer-discovery",
+  "stats": {
+    "evmDescriptors": { "totalContracts": 800, "totalSelectors": 2957 },
+    "evmSignedBlobs": { "totalBlobs": 2933, "verified": 2933 },
+    "tokenSignatures": { "totalSigned": 8730, "verified": 8730 },
+    "cosmosMsgs": { "totalMsgTypes": 28 },
+    "assets": { "total": 29923, "native": 36, "top500": 8730 },
+    "chains": 711
+  }
+}
+```
+
+**`GET /api/v2/manifest`** — everything a client needs to verify signed blobs:
+```json
+{
+  "success": true,
+  "data": {
+    "version": 1,
+    "publicKey": "0218621d9c14473458713bd3e672e53480aa7032ca9b67356395e88709bb45226a",
+    "publicKeyBytes": [2, 24, 98, ...],
+    "derivationPath": "m/13'/44358944'/1285410994'/2003068762'/1451542600'",
+    "blobFormats": { ... },
+    "verification": "All blobs: SHA256(payload) → secp256k1 verify(sig, digest, publicKey)"
+  }
+}
+```
+
+---
+
+### EVM Descriptors (Clear-Signing)
+
+| Method | Path | Description |
+|--------|------|-------------|
+| GET | `/api/v2/descriptors/lookup/:networkId/:contractAddress/:selector` | **Primary** — look up method + signed blob |
+| GET | `/api/v2/descriptors/contract/:networkId/:contractAddress` | Full contract descriptor |
+| GET | `/api/v2/descriptors/dapp/:dappSlug` | All contracts for a dApp |
+| GET | `/api/v2/descriptors/chain/:networkId?page=0&limit=50` | List contracts on a chain |
+
+**`GET /api/v2/descriptors/lookup/eip155:1/0xd37bbe5744d730a1d98d8dc97c42f0ca46ad7146/0x1fece7b4`**
+
+This is the main clear-signing endpoint. Returns method info + pre-signed blob:
+
+```json
+{
+  "success": true,
+  "data": {
+    "networkId": "eip155:1",
+    "chainId": 1,
+    "contractAddress": "0xd37bbe5744d730a1d98d8dc97c42f0ca46ad7146",
+    "contractName": "THORChain_Router",
+    "dappName": "THORChain",
+    "dappSlug": "thorchain",
+    "selector": "0x1fece7b4",
+    "method": "deposit",
+    "plugin": "THORChain",
+    "erc20OfInterest": ["asset"],
+    "parser": null,
+    "signedBlob": "AQAAAAEAz3u+V0TXMKHZjY3JfELwykattRQAAR/s57QAAAA...==",
+    "source": "keepkey"
+  }
+}
+```
+
+**Fallback behavior:** If the contract isn't in the descriptor registry, the endpoint automatically falls back to the universal selector index and returns `{ source: "universal-selector" }`.
+
+---
+
+### Universal Selectors
+
+| Method | Path | Description |
+|--------|------|-------------|
+| GET | `/api/v2/selectors/:selector` | Look up function signature by 4-byte selector |
+
+**`GET /api/v2/selectors/0xa9059cbb`**
+```json
+{
+  "success": true,
+  "data": [{
+    "signature": "transfer(address,uint256)",
+    "methodName": "transfer",
+    "argTypes": ["address", "uint256"],
+    "argNames": ["to", "amount"],
+    "standard": "ERC-20"
+  }]
+}
+```
+
+Covers: ERC-20, ERC-721, ERC-1155, ERC-4626, Uniswap V2/V3, WETH, Multicall, governance, staking, THORChain/Maya routers, + 417 selectors extracted from Ledger descriptors.
+
+---
+
+### Cosmos Message Types
+
+| Method | Path | Description |
+|--------|------|-------------|
+| GET | `/api/v2/cosmos/msgs` | All known msg types with signed blobs |
+| GET | `/api/v2/cosmos/msgs/:typeUrl` | Specific msg type |
+
+**`GET /api/v2/cosmos/msgs`** — returns 28 msg types across:
+- **Cosmos SDK:** MsgSend, MsgDelegate, MsgUndelegate, MsgVote, IBC Transfer, etc.
+- **THORChain:** MsgDeposit, MsgSend
+- **MayaChain:** MsgDeposit, MsgSend
+- **Osmosis:** MsgSwapExactAmountIn, MsgJoinPool, MsgExitPool, MsgLockTokens, etc.
+
+Each entry includes `signedBlob` (base64) for on-device verification.
+
+---
+
+### Token Signatures
+
+| Method | Path | Description |
+|--------|------|-------------|
+| GET | `/api/v2/tokens/signature/:caip` | Signed metadata for a single token |
+| POST | `/api/v2/tokens/signatures` | Batch lookup (up to 500) |
+
+**`GET /api/v2/tokens/signature/eip155%3A1%2Fslip44%3A60`**
+```json
+{
+  "success": true,
+  "caip": "eip155:1/slip44:60",
+  "signedBlob": "ARAIZWlwMTU1OjEDRVRIEgVFdGhlcmV1bWb..."
+}
+```
+
+**`POST /api/v2/tokens/signatures`**
+```json
+// Request
+{ "caips": ["eip155:1/slip44:60", "bip122:000000000019d6689c085ae165831e93/slip44:0"] }
+
+// Response
+{ "success": true, "data": { "eip155:1/slip44:60": "ARAIZWl...", "bip122:...": "ARAIYWJ..." } }
+```
+
+8,730 tokens signed (36 native + top CoinGecko tokens).
+
+---
+
+### Chains
+
+| Method | Path | Description |
+|--------|------|-------------|
+| GET | `/api/v2/chains` | All 711 chains |
+| GET | `/api/v2/chains/:chainId` | Chain by CAIP-2 ID |
+
+**`GET /api/v2/chains/eip155%3A1`**
+```json
+{
+  "success": true,
+  "data": {
+    "chainId": "eip155:1",
+    "name": "Ethereum",
+    "evmChainId": 1,
+    "slip44": 60,
+    "isEVM": true,
+    "nativeCurrency": { "symbol": "ETH", "name": "Ethereum", "decimals": 18 },
+    "explorer": "https://etherscan.io",
+    "thorchainId": "ETH",
+    "mayachainId": "ETH",
+    "ledgerChainName": "ethereum",
+    "knownAssetCount": 11095
+  }
+}
+```
+
+---
+
+### Assets
+
+| Method | Path | Description |
+|--------|------|-------------|
+| GET | `/api/v2/assets/native` | 36 native gas tokens |
+| GET | `/api/v2/assets/top500` | 8,730 top tokens |
+| GET | `/api/v2/assets/:caip` | Single asset by CAIP |
+| GET | `/api/v2/search?q=usdc&limit=50` | Search by symbol/name |
+
+---
+
+### Bulk Exports
+
+For clients that want to sync everything locally and do lookups client-side:
+
+| Method | Path | Size | Description |
+|--------|------|------|-------------|
+| GET | `/api/v2/bulk/evm-signed-blobs` | 0.7 MB | All 2,933 EVM signed blobs |
+| GET | `/api/v2/bulk/evm-descriptors` | 2.7 MB | All 800 contracts raw |
+| GET | `/api/v2/bulk/token-signatures` | 2.0 MB | All 8,730 token sigs |
+| GET | `/api/v2/bulk/universal-selectors` | 77 KB | All 462 selectors |
+| GET | `/api/v2/bulk/cosmos-msgs` | 9 KB | All 28 cosmos msg types |
+| GET | `/api/v2/bulk/chains` | 223 KB | All 711 chains |
+
+---
+
+## Verification
+
+All signed blobs follow the same verification pattern:
+
+```
+blob = base64decode(signedBlob)
+payload = blob[0 .. blob.length - 65]
+sig     = blob[blob.length - 65 .. blob.length - 1]   // 64 bytes (r + s)
+recovery = blob[blob.length - 1]                       // 27 or 28
+
+digest = SHA256(payload)
+valid  = secp256k1_verify(sig, digest, publicKey)
+```
+
+**Public key** (compressed, 33 bytes):
+```
+0218621d9c14473458713bd3e672e53480aa7032ca9b67356395e88709bb45226a
+```
+
+Fetch once from `GET /api/v2/manifest` and cache.
+
+### Blob Formats
+
+**EVM selector blob** (from `serializeMetadata` in pioneer-insight):
+```
+version(1) + chainId(4 BE) + contractAddress(20) + selector(4) + txHash(32) +
+methodNameLen(2 BE) + methodName(var) + numArgs(1) + [args...] +
+classification(1) + timestamp(4 BE) + keyId(1) + sig(64) + recovery(1)
+```
+
+**Token blob:**
+```
+version(1) + type(1: 0x10=native, 0x11=token) + chainIdLen(1) + chainId(var) +
+contractAddr(0 or 20) + symbolLen(1) + symbol(var) + decimals(1) +
+nameLen(2 BE) + name(var) + timestamp(4 BE) + keyId(1) + sig(64) + recovery(1)
+```
+
+**Cosmos msg blob:**
+```
+version(1) + type(1: 0x20) + typeUrlLen(2 BE) + typeUrl(var) +
+nameLen(1) + name(var) + numChains(1) + [chainIdLen(1) + chainId(var)]... +
+numFields(1) + [fieldNameLen(1) + fieldName(var) + displayAs(1)]... +
+timestamp(4 BE) + keyId(1) + sig(64) + recovery(1)
+```
+
+---
+
+## Integration Example (KeepKey Vault)
+
+```typescript
+// 1. Fetch manifest once at startup
+const { data: manifest } = await fetch('/api/v2/manifest').then(r => r.json());
+const pubkey = new Uint8Array(manifest.publicKeyBytes);
+
+// 2. Before signing an EVM tx, look up the descriptor
+const { data } = await fetch(
+  `/api/v2/descriptors/lookup/eip155:1/${contractAddress}/${calldata.slice(0, 10)}`
+).then(r => r.json());
+
+if (data.signedBlob) {
+  // 3. Send signed blob to KeepKey firmware for on-device display
+  const blob = Uint8Array.from(atob(data.signedBlob), c => c.charCodeAt(0));
+  // firmware verifies blob internally using embedded public key
+  await wallet.ethSignTx({ ..., metadata: blob });
+}
+
+// 4. Token identity verification
+const { signedBlob } = await fetch(
+  `/api/v2/tokens/signature/${encodeURIComponent(tokenCaip)}`
+).then(r => r.json());
+```
+
+---
+
+## Data Sources
+
+| Source | Data | Count |
+|--------|------|-------|
+| [ledger-asset-dapps](https://github.com/LedgerHQ/ledger-asset-dapps) | EVM contract selectors + parsers | 794 contracts |
+| KeepKey custom | THORChain + Maya routers | 6 contracts |
+| Hardcoded standards | ERC-20/721/1155/4626, Uniswap, governance | 45 selectors |
+| Extracted from Ledger | Method names from imported descriptors | 417 selectors |
+| CoinGecko mapping | Token metadata (symbol, decimals) | 8,730 tokens |
+| Manual curation | Cosmos SDK, THORChain, Maya, Osmosis msgs | 28 msg types |
+
+## Scripts
+
+```bash
+# Import Ledger descriptors (requires local clone of ledger-asset-dapps)
+node scripts/import-ledger-descriptors.js --ledger-path /path/to/ledger-asset-dapps
+
+# Build universal selector index
+node scripts/import-4byte-selectors.js
+
+# Sign everything (requires INSIGHT_MNEMONIC)
+INSIGHT_MNEMONIC="..." node scripts/sign-all.mjs
+
+# Full pipeline
+node scripts/import-ledger-descriptors.js && node scripts/import-4byte-selectors.js && pnpm run sign
+
+# Serve locally
+pnpm run serve
+```

package/HANDOFF-coingecko-mapping-coverage.md

@@ -0,0 +1,113 @@
+# Handoff — `coingecko-mapping.json` is stale and symbol-matched (Solana/TRON/TON 0% covered)
+
+**Date:** 2026-06-15
+**From:** keepkey-website-v7 (GEO/SEO asset-page work)
+**To:** pioneer-discovery maintainers
+**Severity:** High — blocks using the CoinGecko mapping as a logical-asset / trust key, and silently omits the chains KeepKey markets most.
+
+---
+
+## TL;DR
+
+`src/coingecko-mapping.json` (CAIP → CoinGecko coin id) is **7 months stale** and was built by **symbol+name string matching with no contract addresses**. As a result:
+
+- **Solana: 0 of 5,023 assets mapped. TRON: 0/3. TON: 0/1. XRPL token: 0/1.** Every real Solana token — USDT, USDC, BONK, JUP, WIF, JTO, PYTH, RAY — is unmapped, with the correct mint addresses sitting right there in `generatedAssetData.json`.
+- Because matching is by symbol, the mapping **cannot be a trust signal**: a typosquat ("Tenter USD T", symbol `USDT`) is indistinguishable from real Tether, and real `WIF` (mint `EKpQGSJtjMFqKZ…`) is indistinguishable from `HAMSTER WIF HAT`.
+
+These are the exact chains the marketing/GEO copy leads with ("Solana, TON, TRON"). Any consumer that uses CoinGecko-mapping membership to group multi-chain deployments and filter scams (which is keepkey.com's planned asset-page model) would **drop the entire Solana ecosystem** and **fail to separate real from scam** on it.
+
+---
+
+## Evidence (reproducible)
+
+Coverage by CAIP namespace, measured against the published package + current mapping:
+
+| Namespace | Total assets | Mapped | Coverage |
+|---|---:|---:|---:|
+| eip155 (EVM) | 24,877 | 9,197 | 37% |
+| **solana** | **5,023** | **0** | **0%** |
+| cosmos | 9 | 4 | 44% |
+| bip122 (BTC family) | 7 | 5 | 71% |
+| **tron** | **3** | **0** | **0%** |
+| **ton** | **1** | **0** | **0%** |
+| ripple | 1 | 1 | 100% |
+| binance / xrpl | 2 | 0 | 0% |
+
+By token standard: `token:` (Solana SPL) **0 / 5,021**; `erc20` 34%; `bep20` 43%; `slip44` (natives) 76%.
+
+Major Solana tokens present in `assetData` but unmapped (real mints):
+```
+USDT  Es9vMFrzaCERmJfrF4H2FYD4KCoNkY11McCe8BenwNYB   (real Tether mint)
+USDC  EPjFWdd5AufqSSqeM2qN1xzybapC8G4wEGGkZwyTDt1v   (real Circle mint)
+BONK  DezXAZ8z7PnrnRJjz3wXBoRgixCa6xjnB7YaB1pPB263
+JUP   JUPyiwrYJFskUPiHa7hkeR8VUtAeFoSYbKedZNsDvCN
+WIF   EKpQGSJtjMFqKZ…   (real dogwifhat — alongside scams "HAMSTER WIF HAT", "Wif  SecondChance")
+JTO, PYTH, RAY, WEN, POPCAT … all unmapped
+```
+
+---
+
+## Root cause 1 — Stale (never regenerated after Solana/TRON/TON were added)
+
+`scripts/.coingecko-mapping-stats.json`:
+```
+startTime: 2025-11-01T03:18:17Z   totalAssets: 14167   mapped: 10168 (71.77%)
+solana entries in unmapped list: 0   ← Solana assets did not exist in the registry yet
+```
+- `src/coingecko-mapping.json` mtime: **2026-01-16**
+- `src/generatedAssetData.json` mtime: **2026-06-07**, now **29,923** assets
+
+~15,700 assets (Solana/TRON/TON SPL + new EVM tokens) were added to the registry **after** the mapping was last generated. The mapping has never caught up.
+
+## Root cause 2 — Symbol+name matching, no contract addresses
+
+`scripts/generate-coingecko-mapping.js` calls `GET /coins/list` (returns only `{id, symbol, name}`) and matches via `findBestMatch()` on `symbol` then `name` string equality/substring (lines 43–96). It never looks at contract/mint addresses. Consequences:
+- Ambiguous by construction: any symbol with multiple CoinGecko coins falls back to "first match" (line 95) — wrong for scams and bridged variants.
+- Cannot verify a specific deployment is the *real* one, so the mapping is unsafe as a scam filter even where it has coverage.
+
+---
+
+## Recommended fix
+
+### 1. (Immediate) Regenerate against the current `generatedAssetData.json`
+A re-run today would symbol-match the Solana majors and lift coverage off 0% — a stopgap. But do **not** stop here; symbol matching will also map scams. Pair with #2.
+
+### 2. (Correct fix) Match by contract address via `include_platform=true`
+Use `GET /coins/list?include_platform=true`, which returns per-coin `platforms`:
+```json
+{ "id":"tether","symbol":"usdt","name":"Tether",
+  "platforms":{ "ethereum":"0xdac17f95…","solana":"Es9vMFrzaCERm…","binance-smart-chain":"0x55d39833…","tron":"TR7NHqje…" } }
+```
+Build a reverse index `(platform, lowercasedAddress) → coinId`, then for each CAIP derive `(platform, address)` and look it up. This is **exact and scam-proof**: real Solana USDC (`EPjFWdd5…`) maps to `usd-coin`; the typosquat doesn't map to anything.
+
+Needs a CAIP-namespace → CoinGecko-platform-id table, e.g.:
+```
+eip155:1→ethereum  eip155:56→binance-smart-chain  eip155:8453→base  eip155:137→polygon-pos
+eip155:42161→arbitrum-one  solana:5eykt4…→solana  tron:…→tron  ton:…→the-open-network
+```
+- Keep symbol+name only as a **fallback for natives** (`slip44`), which have no contract and already map well (76%).
+- `include_platform=true` is a heavier payload / stricter rate limit — may need a CoinGecko API key (Demo/Pro). Flag for the team.
+
+### 3. Stop it going stale — add a guard
+Regenerate the mapping whenever `generatedAssetData.json` changes, or add a CI check in `validate-baseline.js` that fails when `mappedCount / assetCount` drops below a threshold, or when any whole namespace present in `assetData` has 0% coverage. The current 0%-Solana state should have failed a gate.
+
+---
+
+## Decisions needed from the pioneer team
+1. CoinGecko **API tier / key** for `include_platform=true` (rate limits). Pro vs Demo?
+2. Canonical handling of **bridged vs native** variants (CoinGecko gives them distinct ids: `tether` vs `bridged-usdt`). Map to the true id per contract (correct), or collapse bridged into the parent for display? (Consumer-side concern, but affects what id you store.)
+3. Authoritative **platform-id table** for all chains in the registry (above is a starter; 711 distinct chainIds exist — most are dust, but confirm the ~20 real ones).
+4. Republish cadence + version so consumers (keepkey.com pinned `^10.0.8`) can rely on coverage.
+
+---
+
+## Why this matters to keepkey.com specifically
+The new asset-page model keys logical-asset grouping and scam-filtering on CoinGecko-mapping membership. With Solana at 0%, "exclude unmapped" would erase KeepKey's flagship-marketed Solana support (USDT/USDC/BONK/JUP/WIF/…) from the site, while "include unmapped" would let Solana scams through. Neither is acceptable until the mapping covers Solana/TRON/TON by contract address. **This handoff is the upstream unblock for that work.**
+
+## Reproduce
+From any project with the package installed:
+```
+node -e 'const m=require("@pioneer-platform/pioneer-discovery");const cg=m.coingeckoMapping,{assetData}=m;
+const ns=id=>id.split(":")[0];const b={};for(const id in assetData){const n=ns(id);(b[n]=b[n]||{t:0,m:0});b[n].t++;if(cg[id])b[n].m++;}
+console.log(Object.entries(b).map(([k,o])=>`${k}: ${o.m}/${o.t}`).join("\n"))'
+```