Rework shielded transactions (with crypto in WASM)

[l0r1s]

Summary

Companion PR to opentensor/polkadot-sdk#6 — supersedes #2441.

Reworks the MEV Shield from a store-then-decrypt model to a decrypt-at-proposal model,
with all cryptographic operations running entirely in WASM — no custom host functions required.

Previously, encrypted transactions were stored on-chain (Submissions) and decrypted by the block
author in a separate step, leaving a window where decrypted transactions sat in the pool before
inclusion. Now, encrypted transactions travel through the pool as opaque ciphertext and the block
proposer decrypts them inline during block building, including the inner extrinsic in the same
block. There is no point where a decrypted transaction is visible before it lands in a finalized
block.

Encryption uses ML-KEM-768 + XChaCha20-Poly1305 with per-block ephemeral keys rotated via an
inherent. The node passes the raw decapsulation key bytes to the runtime API, which performs
ML-KEM decapsulation and AEAD decryption in pure WASM — eliminating the need for custom host
functions (stp-io has been removed) and allowing the shield to be activated with a runtime-only
upgrade.

Note: E2E tests have been moved to the setup-e2e branch for ease of review.

Breaking changes for clients

This PR introduces breaking changes that require client-side updates.

Encrypted message format

The ciphertext payload now includes the next public key hash as a prefix. The new wire format is:

key_hash (16 bytes) || kem_len (2 bytes LE) || kem_ct (variable) || nonce (24 bytes) || aead_ct (variable)

• key_hash: xxhash128(NextKey) — the 16-byte hash of the ML-KEM public key used for encryption. Clients must compute this from the NextKey they encrypt against.
• kem_len: 2-byte little-endian length of the KEM ciphertext that follows.
• kem_ct: ML-KEM-768 encapsulation ciphertext (typically 1088 bytes).
• nonce: 24-byte random nonce for XChaCha20-Poly1305.
• aead_ct: The authenticated ciphertext containing the encoded inner extrinsic.

submit_encrypted API change

The old unused commitment parameter has been removed. The new signature is:

submit_encrypted(origin, ciphertext)

New transaction extension: CheckShieldedTxValidity

A new transaction extension validates encrypted messages at two levels:

• Pool validation — rejects ciphertext that cannot be parsed (malformed structure). Returns FailedShieldedTxParsing error.
• In-block validation — the block proposer verifies that key_hash matches either CurrentKey or NextKey. If it doesn't match, the transaction is dropped silently — the block proposer returns an Invalid transaction error without a custom error code because the block proposer does not propagate the inner error code of the extension.

This means clients should expect:

• A clear FailedShieldedTxParsing error if the ciphertext format is wrong.
• A generic Invalid error (no specific code) if the key_hash doesn't match any active key — e.g. when using a stale or wrong key.

Rollout plan

Since all crypto runs in WASM, non-validator nodes do not need a binary upgrade — the shield-related node code (ShieldKeystore, key rotation, proposer decryption) is gated behind role.is_authority and the block proposer. Community nodes receive the new behaviour automatically via the runtime upgrade.

1. Validator node upgrade — validators deploy the new binary. This gives them the ShieldKeystore for key management and rotation, and the block proposer gains the ability to pass decapsulation keys to the runtime.
2. Runtime upgrade — activate the new runtime with pallet-shield and the ShieldApi runtime API. The shield is fully operational once the runtime is upgraded — no host function registration required.

Architecture

All cryptographic operations (ML-KEM-768 decapsulation, XChaCha20-Poly1305 decryption) run
entirely in WASM. The node side is responsible only for key generation, storage, and rotation;
the raw decapsulation key bytes are passed to the runtime API as a parameter.

Operation	Runs in	Details
Key generation & storage	Node (stc-shield)	MemoryShieldKeystore with ML-KEM-768
Key rotation	Node (background task)	Rolls keypairs on own block import
Public key announcement	WASM (inherent)	Pallet stores CurrentKey / NextKey
Shielded tx detection	WASM (runtime API)	try_decode_shielded_tx parses wrapper
ML-KEM decapsulation	WASM (runtime API)	try_unshield_tx — pure ml-kem crate
AEAD decryption	WASM (runtime API)	try_unshield_tx — pure chacha20poly1305 crate
Block building orchestration	Node (proposer)	Calls runtime API, pushes both txs

Changes

pallet-shield (reworked)

• announce_next_key inherent: rotates CurrentKey <- NextKey each block and publishes the next block author's ML-KEM public key
• submit_encrypted extrinsic: accepts an encrypted ciphertext wrapper — the block proposer decrypts and includes the inner extrinsic in the same block
• try_decode_shielded_tx / try_unshield_tx: runtime helpers called by the block proposer via the ShieldApi runtime API
• In-pallet unshield function: performs ML-KEM-768 decapsulation + XChaCha20-Poly1305 AEAD decryption entirely in WASM, using ml-kem and chacha20poly1305 crates directly
• CheckShieldedTxValidity transaction extension: pool validation checks ciphertext structure; block import additionally validates key_hash against CurrentKey/NextKey
• FindAuthors trait for resolving current and next block author
• Migration to clear removed v1 storage (Submissions, KeyHashByBlock)

stp-io (removed)

• The stp-io crate (custom host functions for ML-KEM and AEAD) has been deleted entirely. All crypto now runs in WASM via pallet-shield.

Node integration

• ShieldKeystore created in service.rs and threaded through to the proposer and inherent providers
• ShieldInherentDataProvider added to Aura consensus configuration
• KeyRotationService generates a new ML-KEM keypair on each own-block import
• SubtensorHostFunctions removed from the client executor — no longer needed

Runtime integration

• ShieldApi runtime API implemented — try_unshield_tx accepts dec_key_bytes: Vec<u8> and decrypts in WASM
• CheckShieldedTxValidity added to the transaction extension pipeline

Tests

• Unit: key rotation, try_decode_shielded_tx, try_unshield_tx (with in-WASM decryption), depth-limit protection, inherent creation, CheckShieldedTxValidity extension (key_hash matching, malformed rejection, pool vs in-block source)

E2E (branch: setup-e2e)

14 tests across 3 files covering 3-node and 6-node topologies:

Key rotation (3-node)

• NextKey and CurrentKey are populated and rotate across blocks
• AuthorKeys stores per-author keys

Encrypted transactions (3-node)

• Happy path: wrapper and inner tx included in the same block
• Failed inner tx: wrapper succeeds but inner transfer has no effect
• Malformed ciphertext rejected at pool level
• Wrong key hash not included by block proposer
• Stale key not included after rotation
• Multiple encrypted txs in the same block

Scaling (6-node)

• Network scales to 6 nodes with full peering
• Key rotation continues with more peers
• Encrypted tx works with 6 nodes
• Multiple encrypted txs in the same block with 6 nodes

Edge cases

• CurrentKey fallback: encrypted tx persists across blocks
• Valid ciphertext with invalid inner call