polkadot-cli 1.17.0 → 1.18.0

package/README.md

@@ -20,12 +20,13 @@ Ships with Polkadot and all system parachains preconfigured with multiple fallba
 - ✅ Account management — BIP39 mnemonics, derivation paths, env-backed secrets, watch-only, dev accounts
 - ✅ Named address resolution across all commands
 - ✅ Runtime API calls — `dot polkadot.apis.Core.version`
+- ✅ Raw JSON-RPC calls — `dot polkadot.rpc.system_health`, with discovery via `rpc_methods` and tab-completion
 - ✅ Full-metadata dump — `dot metadata <chain>` emits one JSON blob with pallets, runtime APIs, and transaction extensions (or raw SCALE bytes via `--raw`)
 - ✅ Stale-metadata detection — when a tx or query fails because the runtime upgraded, the CLI tells you exactly which `dot chain update` to run
 - ✅ Chain topology — relay/parachain hierarchy with tree display and auto-detection
 - ✅ Batteries included — all system parachains and testnets already setup to be used
 - ✅ File-based commands — run any command from a YAML/JSON file with variable substitution
-- ✅ Parachain sovereign accounts — derive child and sibling addresses from a parachain ID
+- ✅ Sovereign accounts — store a parachain (child / sibling) or pallet (Treasury, Bounties, NominationPools, …) sovereign as a named watch-only account in one command
 - ✅ Message signing — sign arbitrary bytes with account keypairs for use as `MultiSignature` arguments
 - ✅ Unsigned/authorized transactions — submit governance-authorized calls without a signer (`--unsigned`)
 - ✅ Non-native fee payment — pay tx fees in any asset the chain accepts via `--asset` (asset-hub-style chains)
@@ -323,7 +324,7 @@ dot account add treasury 5GrwvaEF5zXb26Fz9rcQpDWS57CtERHpNehXCPcNoHGKutQY
 dot account add council 0xd43593c715fdd31c61141abd04a99fd6822c8558854ccde39a5684e7a56da27d
 ```
 
-Watch-only accounts appear in `dot account list` with a `(watch-only)` badge and can be inspected and removed like any other account. They cannot be used with `--from` (signing) or as a source for `derive`.
+Watch-only accounts appear in `dot account list` under a dedicated **Watch-only** section and can be inspected and removed like any other account. They cannot be used with `--from` (signing) or as a source for `derive`.
 
 The `add` subcommand is context-sensitive: bare `add <name> <address>` creates a watch-only entry, while `add --secret` or `add --env` imports a keyed account. `dot account import` is reserved for file-based batch import.
 
@@ -363,6 +364,8 @@ Convert between SS58 addresses, hex public keys, and account names. Accepts any
 - **Stored account name** — looks up the public key from the accounts file
 - **SS58 address** — decodes to the underlying public key
 - **Hex public key** (`0x` + 64 hex chars) — encodes to SS58
+- **`--pallet-id <id>`** — derives a pallet sovereign address without saving it (script-friendly; nothing persists)
+- **`--parachain <id> --parachain-type <child|sibling>`** — derives a parachain sovereign address without saving it
 
 ```bash
 dot account inspect alice
@@ -388,7 +391,8 @@ dot account inspect alice --json
 #   "publicKey": "0xd43593c715fdd31c61141abd04a99fd6822c8558854ccde39a5684e7a56da27d",
 #   "ss58": "5GrwvaEF5zXb26Fz9rcQpDWS57CtERHpNehXCPcNoHGKutQY",
 #   "prefix": 42,
-#   "name": "Alice"
+#   "name": "Alice",
+#   "kind": "dev"
 # }
 ```
 
@@ -400,11 +404,52 @@ dot account inspect alice
 # Account Info
 #
 #   Name:        Alice
+#   Kind:        dev
 #   Public Key:  0xd43593c715fdd31c61141abd04a99fd6822c8558854ccde39a5684e7a56da27d
 #   SS58:        5GrwvaEF5zXb26Fz9rcQpDWS57CtERHpNehXCPcNoHGKutQY
 #   Prefix:      42
 ```
 
+The `Kind:` line categorises the account: `dev` (built-in), `signer` (has a secret/env), `watch-only` (raw external address), `pallet sovereign` (derived from a `PalletId`), or `parachain sovereign (child|sibling)` (derived from a parachain ID). For derived sovereigns, an extra `Source:` line shows what the address was derived from. For env-backed signers, an `Env:` line shows the variable; for derived child keys, `Derivation:` shows the path.
+
+##### Stateless sovereign derivation (script-friendly)
+
+Pass `--pallet-id` or `--parachain` / `--parachain-type` to compute a sovereign address **without persisting** anything to `~/.polkadot/accounts.json`. The output shape matches the stored case (same `Kind:` / `Source:` / SS58 / public key + same `--json` schema), but no `Name:` line and nothing in `dot account list` afterwards. Use this in scripts when you just need the address:
+
+```bash
+# Polkadot Treasury — pallet sovereign on prefix 0
+dot account inspect --pallet-id py/trsry --prefix 0
+# Output:
+# Account Info
+#
+#   Kind:        pallet sovereign
+#   Public Key:  0x6d6f646c70792f74727372790000000000000000000000000000000000000000
+#   SS58:        13UVJyLnbVp9RBZYFwFGyDvVd1y27Tt8tkntv6Q7JVPhFsTB
+#   Source:      PalletId py/trsry (0x70792f7472737279)
+#   Prefix:      0
+
+# Hex form works the same
+dot account inspect --pallet-id 0x70792f7472737279 --prefix 0
+
+# Parachain sovereigns (type is required — child = on relay, sibling = on another parachain)
+dot account inspect --parachain 1004 --parachain-type child
+
+# Pipeline: just the SS58
+SS58=$(dot account inspect --pallet-id py/trsry --prefix 0 --json | jq -r .ss58)
+
+# JSON shape includes a `source` object describing the derivation
+dot account inspect --pallet-id py/trsry --json
+# {
+#   "publicKey": "0x6d6f646c70792f74727372790000000000000000000000000000000000000000",
+#   "ss58":      "5EYCAe5ijiYfyeZ2JJCGq56LmPyNRAKzpG4QkoQkkQNB5e6Z",
+#   "prefix":    42,
+#   "kind":      "pallet sovereign",
+#   "source":    { "kind": "pallet", "palletId": "py/trsry", "palletIdHex": "0x70792f7472737279" }
+# }
+```
+
+Constraints (will error): cannot combine a positional input with derivation flags; `--pallet-id` and `--parachain` are mutually exclusive; `--parachain` requires `--parachain-type child|sibling`; `--show-secret` doesn't apply (derived sovereigns have no key).
+
 #### Reveal the sr25519 private key
 
 For provisioning another signer (e.g. a server that expects a raw hex private key in an env var), add `--show-secret` to print the **64-byte sr25519 expanded secret** as `0x`-prefixed hex:
@@ -426,7 +471,7 @@ dot account add ci-signer --env MY_SECRET
 
 `--secret` and `--env` are mutually exclusive. Use `dot account add` for single-account imports; `dot account import` is reserved for file-based batch import.
 
-The secret is never written to disk. At signing time, the CLI reads `$MY_SECRET` and derives the keypair. If the variable is not set, the CLI errors with a clear message. `account list` shows an `(env: MY_SECRET)` badge and resolves the address live when the variable is available.
+The secret is never written to disk. At signing time, the CLI reads `$MY_SECRET` and derives the keypair. If the variable is not set, the CLI errors with a clear message. `account list` annotates env-backed signers with `env $MY_SECRET` in the metadata column and resolves the address live when the variable is available.
 
 #### Derivation paths
 
@@ -448,11 +493,15 @@ dot account derive treasury treasury-staking --path //staking
 
 `derive` copies the source account's secret and applies the given path. It requires both a source name, a new name, and `--path`. Works with env-backed accounts too — the derived account shares the same env var reference.
 
-`account list` shows the derivation path next to the account name:
+`account list` shows derivation paths and env sources on tree-style continuation lines (label names mirror the `--flag` that sets them):
 
 ```
-  treasury-staking (//staking)  5FHneW46...
-  ci-signer (//ci) (env: MY_SECRET)  5EPCUjPx...
+Signers
+  treasury-staking  5FHneW46...
+     └─ path: //staking
+  ci-signer         5EPCUjPx...
+     ├─ path: //ci
+     └─ env:  $MY_SECRET
 ```
 
 **Supported secret formats for import:**
@@ -532,7 +581,7 @@ dot polkadot.apis.Core
 dot apis Core --chain polkadot
 ```
 
-This works for all categories (`query`, `tx`, `const`, `events`, `errors`, `apis`, `extensions`). When passing positional method arguments, keep `Pallet` and `Item` either fully dot-joined (`query.System.Account 5Grw...`) or fully space-separated (`query System Account 5Grw...`) — mixing the two (`query System.Account 5Grw...`) does not work because the second arg gets parsed as a pallet name.
+This works for all categories (`query`, `tx`, `const`, `events`, `errors`, `apis`, `extensions`, `rpc`). When passing positional method arguments, keep `Pallet` and `Item` either fully dot-joined (`query.System.Account 5Grw...`) or fully space-separated (`query System Account 5Grw...`) — mixing the two (`query System.Account 5Grw...`) does not work because the second arg gets parsed as a pallet name.
 
 ### Query storage
 
@@ -963,6 +1012,74 @@ The list view tags each entry:
 
 The detail view shows the extension's value type, its `additionalSigned` type, and a ready-to-adapt `--ext` snippet for custom extensions. Use this to discover what `--ext` payload a chain expects before submitting a `dot tx` command.
 
+### Raw JSON-RPC
+
+Substrate nodes expose a JSON-RPC surface that lives outside runtime metadata: `system_*` (sync state, peers, version), `chain_*` (blocks, headers, finalized head), `state_*` (raw storage, key iteration, runtime version), `author_*` (mempool, key management), `payment_*` (fee estimation), consensus families (`babe_*`, `grandpa_*`, `mmr_*`, `beefy_*`), and the new spec families (`chainSpec_v1_*`, `archive_v1_*`, `rpc_methods`). The `rpc` category exposes them all.
+
+Methods are discovered per-chain via the standard `rpc_methods` JSON-RPC call and cached at `~/.polkadot/chains/<chain>/rpc-methods.json`. The set of available methods depends on the node, not the chain — an archive node adds `archive_v1_*`, validators may add `babe_epochAuthorship`, dev nodes add `dev_newBlock`, and `--rpc-methods safe` strips writes.
+
+```bash
+# List all methods the node exposes, grouped by family
+dot polkadot.rpc
+# Output:
+# RPC methods on polkadot (129)
+#
+# system (20)
+#   system_health  Node sync state (peers, isSyncing, shouldHavePeers).
+#   system_version  Node software version string.
+#   system_chain  Chain name as reported by the node.
+#   ...
+# chain (19)
+#   chain_getBlock  Full block (header + extrinsics) by hash.
+#   chain_getFinalizedHead  Hash of the latest finalized head.
+#   ...
+
+# Call a method
+dot polkadot.rpc.system_health
+# Output:
+# {
+#   "peers": 131,
+#   "isSyncing": false,
+#   "shouldHavePeers": true
+# }
+
+# Positional args (parsed by the same heuristic as tx args)
+dot polkadot.rpc.chain_getBlockHash 1000
+# Output:
+# "0xcf36a1e4a16fc579136137b8388f35490f09c5bdd7b9133835eba907a8b76c30"
+
+# Show curated info for a known method
+dot polkadot.rpc.author_insertKey --help
+# Output:
+# author_insertKey
+#   ⚠️  WRITE / state-changing
+#   Insert a key into the node keystore.
+#
+#   Family: author
+#   Args:   <keyType: string> <suri: string> <publicKey: hex>
+
+# Refresh the cached method list (after a node upgrade)
+dot polkadot.rpc --refresh
+
+# JSON output works on any method
+dot polkadot.rpc.system_health --json
+```
+
+`dot chain add` and `dot chain update` automatically populate the method cache, and `dot chain info <name>` shows a per-family breakdown. Tab completion sources from the cache, so methods on a custom chain start completing the moment you've added or updated it.
+
+About 50 well-known methods carry curated metadata (description, named args, `⚠️ WRITE` tag for state-changing calls). Any other method the node reports is callable via raw passthrough — args are forwarded as-is to the JSON-RPC `params` array.
+
+Subscription methods (`*_subscribe*`, `chainHead_v1_follow`, `transaction_v1_*`) appear in tab-completion but error out as one-shots — they need a long-running follow session that doesn't fit a single CLI invocation:
+
+```bash
+dot polkadot.rpc.chain_subscribeAllHeads
+# Error: "chain_subscribeAllHeads" is a subscription method (requires a follow
+# session) and is not callable as a one-shot. Use a long-running client for
+# streaming RPC.
+```
+
+The `rpc` category is **flat** — there's no pallet level. The form is `[chain.]rpc.<method_name>`, where `<method_name>` keeps its underscores in a single segment (e.g. `polkadot.rpc.chain_getBlock`).
+
 ### Submit extrinsics
 
 Build, sign, and submit transactions. Pass a `Pallet.Call` with arguments, or a raw SCALE-encoded call hex (e.g. from a multisig proposal or governance). Both forms display a decoded human-readable representation of the call.
@@ -1579,50 +1696,91 @@ Output shows the crypto type, message bytes in hex, raw signature, and an `Enum`
 
 Use `--type` to select the signature algorithm (default: `sr25519`). Run `dot sign` with no arguments to see usage and examples.
 
-### Parachain sovereign accounts
+### Sovereign accounts (parachain & pallet)
+
+`dot account add` accepts derivation flags that compute a deterministic sovereign address and store it as a named watch-only account — reusable in `--from` (for `--unsigned` flows), as a tx argument, and in `dot account list`. Runs offline; no chain connection required.
 
-Derive the sovereign account addresses for a parachain. These are deterministic accounts derived from a parachain ID — no chain connection required.
+Two kinds of sovereign:
 
-- **Child** accounts represent a parachain on the relay chain (prefix `"para"`)
-- **Sibling** accounts represent a parachain on another parachain (prefix `"sibl"`)
+- **Pallet sovereign** — every FRAME pallet that holds funds (Treasury, Bounties, Crowdloan, Society, NominationPools, ChildBounties, …) declares an 8-byte `PalletId`. The 32-byte account ID is `b"modl"` (4 bytes) + `palletId` (8 bytes) + 20 zero bytes. (`AccountIdConversion::into_account_truncating` with `PalletId::TYPE_ID = b"modl"` from `frame_support`.)
+- **Parachain sovereign** — every parachain has a `child` account (its account on the relay chain, `b"para"` prefix) and a `sibling` account (its account on another parachain, `b"sibl"` prefix). Both are 32-byte IDs of the form `prefix (4 bytes)` + `paraId as LE u32 (4 bytes)` + 24 zero bytes.
 
 ```bash
-# Show both child and sibling accounts
-dot parachain 1000
+# Pallet sovereign — ASCII PalletId
+dot account add Treasury --pallet-id py/trsry
 # Output:
-# Parachain 1000 — Sovereign Accounts
+# Account Added (watch-only)
 #
-#   Child:
-#     Public Key:  0x70617261e8030000000000000000000000000000000000000000000000000000
-#     SS58:        5Ec4AhPZk8STuex8Wsi9TwDtJQxKqzPJRCH7348Xtcs9vZLJ
-#     Prefix:      42
-#
-#   Sibling:
-#     Public Key:  0x7369626ce8030000000000000000000000000000000000000000000000000000
-#     SS58:        5Eg2fntNprdN3FgH4sfEaaZhYtddZQSQUqvYJ1f2mLtinVhV
-#     Prefix:      42
+#   Name:    Treasury
+#   Address: 5EYCAe5ijiYfyeZ2JJCGq56LmPyNRAKzpG4QkoQkkQNB5e6Z
+#   Source:  pallet py/trsry (0x70792f7472737279)
 
-# Show only the child (relay chain) account
-dot parachain 2004 --type child
+# Pallet sovereign — 0x-prefixed hex (16 hex chars)
+dot account add Bounties --pallet-id 0x70792f626f756e74
 
-# Show only the sibling (parachain-to-parachain) account
-dot parachain 2004 --type sibling
+# Parachain sovereign — child (relay-chain side) — type is required
+dot account add People --parachain 1004 --parachain-type child
+# Output:
+# Account Added (watch-only)
+#
+#   Name:    People
+#   Address: 5Ec4AhPaYcfBz8fMoPd4EfnAgwbzRS7np3APZUnnFo12qEYk
+#   Source:  parachain 1004 (child sovereign)
 
-# Use Polkadot SS58 prefix (default: 42)
-dot parachain 1000 --prefix 0
+# Parachain sovereign — sibling (XCM peer side)
+dot account add People-Sibling --parachain 1004 --parachain-type sibling
 
-# JSON output
-dot parachain 1000 --json
-# Output:
+# JSON output includes a `derivation` object describing the source
+dot account add Bnt --pallet-id py/bount --json
 # {
-#   "paraId": 1000,
-#   "prefix": 42,
-#   "child":   { "publicKey": "0x70617261...", "ss58": "5Ec4AhPZ..." },
-#   "sibling": { "publicKey": "0x7369626c...", "ss58": "5Eg2fntN..." }
+#   "name": "Bnt",
+#   "address": "5EYCAe5ijiYdYTM8d3VytEARdH7dFp4rdCPpAsPXrfopdm7d",
+#   "watchOnly": true,
+#   "derivation": {
+#     "kind": "pallet",
+#     "palletId": "py/bount",
+#     "palletIdHex": "0x70792f626f756e74"
+#   }
 # }
+
+# Stored sovereigns appear alongside other accounts
+dot account list
+```
+
+#### Discovering a chain's PalletId
+
+Pallets that need a sovereign account expose their `PalletId` as a runtime constant. Read it via the `const` category and feed the hex straight into `--pallet-id`:
+
+```bash
+# Pre-req: metadata cached for the chain (e.g. `dot chain update polkadot`)
+dot polkadot.const.Treasury.PalletId
+# Output (JSON-quoted hex):
+# "0x70792f7472737279"
+
+# Pipe into the add command (strip JSON quotes with tr)
+dot account add Treasury --pallet-id "$(dot polkadot.const.Treasury.PalletId | tr -d '"')"
 ```
 
-Run `dot parachain` with no arguments to see usage and examples.
+There is no central registry of "well-known" PalletIds — each runtime author picks the 8 bytes when wiring up a pallet's `Config`. The chain's metadata is the authoritative source for that chain's values.
+
+#### Constraints
+
+- `--parachain` requires `--parachain-type child|sibling` (no implicit default — picking the wrong one silently produces a different address).
+- `--parachain` and `--pallet-id` are mutually exclusive.
+- A positional address (`dot account add foo <ss58>`) cannot be combined with derivation flags.
+- Derivation flags cannot be combined with `--secret` or `--env` — a derived sovereign has no signing key.
+
+#### Legacy `dot parachain` command (deprecated)
+
+The standalone `dot parachain <paraId>` command from earlier releases is **still available for backward compatibility** and now prints a deprecation warning to stderr. Stdout output is unchanged, so existing pipes (e.g. `dot parachain 1000 --json | jq`) keep working. Migrate to `dot account inspect --parachain <id> --parachain-type <child|sibling>` at your convenience — it will be removed in a future release ([#208](https://github.com/peetzweg/polkadot-cli/issues/208)).
+
+```bash
+# Old (deprecated, still works — emits stderr warning)
+dot parachain 1000 --type child --json
+
+# New
+dot account inspect --parachain 1000 --parachain-type child --json
+```
 
 ### Bandersnatch member keys