feat(sdk): add hardened encrypted txMetadata documents

[shumkov]

Issue being fixed or feature implemented

Android clients can now create and fetch legacy-compatible encrypted txMetadata through the shared Rust wallet path without depending on the retired dashj Platform stack. This is the owned replacement for #4186 because its contributor branch is not maintainable by us; the first 21 commits preserve bfoss765 as author and replay source head 6f7abba onto v4.2-dev.

Related: #4086, #4091, #4186, #4195

What was done?

• Added the txMetadata envelope, key derivation, create/broadcast, paginated fetch/decrypt, C FFI, JNI, and host-thin Kotlin APIs. The Rust core remains the source of truth for supported wire versions and payload bounds.
• Made external-signable wallets derive through the mnemonic resolver while scrubbing resolved master material and plaintext native copies at the earliest safe point. Teardown gating prevents borrowed native handles from being freed during Kotlin calls.
• Hardened the boundary behavior: invalid inputs fail before signer, resolver, allocation, or network work; output pointers are null-published before fallible work; runtime startup can be retried; worker failures use bounded stable messages; and Kotlin's typed invalid-parameter error remains compatible with callers matching the existing Generic fallback.
• Kept successful-device logs free of identity-correlating identifiers, caller-controlled document types, raw handles, plaintext, and raw native error bodies.
• Pinned wire compatibility with network-free dashj/JVM vectors and a captured encrypted blob produced by a stock dash-wallet 11.9 testnet install. The manual testnet fetch helper no longer contains the throwaway recovery phrase or prints decrypted material.

Stack sequencing: #4195's allocator work will be replaced by U3 on top of this branch, followed by the Swift surface and the shared decrypt-output lifetime fix. Merge the stack in order and do not cut a v4.2 release between this foundation and the decrypt-output fix; this PR still returns decrypted payloads as base64 JSON through a normal CString.

How Has This Been Tested?

• Red-to-green regressions demonstrated that the previous code reached resolver/context work before rejecting unsupported versions, retained failed runtime initialization, copied oversized JNI payloads, and broke Kotlin Generic fallback matching. Each now fails before the protected operation or preserves compatibility as intended.
• platform-wallet library tests: 511 passed.
• platform-wallet-ffi library tests: 221 passed.
• rs-unified-sdk-jni library tests: 49 passed.
• Kotlin debug assembly and unit tests: 188 passed, with no failures or skips.
• cargo fmt --all -- --check and git diff --check passed.
• cargo clippy for all targets in the three touched Rust crates completed successfully; 13 existing warnings remain in unrelated code and none are introduced in the touched U2 files.
• Android NDK r28b x86_64 dev build with ABI verification passed: 227 JNI exports, one anchored C ABI export, and zero misaligned LOAD segments.
• The ignored testnet integration target compiles, and the real-install legacy vector runs network-free in the library suite. The live testnet check was not run.
• Android instrumented tests were not run because this machine has no installed AVD or connected device.

Independent correctness, FFI/security, and Kotlin compatibility reviewers approved the final diff after their must-fixes were folded in.

Breaking Changes

None. The new encrypted-document surface and error subtype are additive; PlatformWallet.InvalidParameter continues to satisfy existing PlatformWallet.Generic matching and reports native code 2.

Checklist:

• I have performed a self-review of my own code
• I have commented my code, particularly in hard-to-understand areas
• I have added or updated relevant unit/integration/functional/e2e tests
• I have added "!" to the title and described breaking changes in the corresponding section if my code contains any
• I have made corresponding changes to the documentation if needed

For repository code-owners and collaborators only

• I have assigned this pull request to a milestone

Summary by CodeRabbit

• New Features
  • Added support for creating and retrieving encrypted documents.
  • Encrypted document retrieval decrypts matching records and supports timestamp-based queries.
  • Added compatible transaction-metadata encryption with wallet-based key handling.
• Bug Fixes
  • Added clearer invalid-parameter errors for unsupported versions and oversized payloads.
  • Improved handling of runtime and worker failures without exposing sensitive data.
• Compatibility
  • Added legacy-compatible encryption and derivation behavior for existing transaction metadata.

[coderabbitai]

📝 Walkthrough

Walkthrough

The PR adds legacy-compatible encrypted transaction-metadata documents. It implements key derivation, encryption, decryption, paginated queries, Rust FFI, JNI and Kotlin APIs, typed validation errors, and failure-handling tests.

Changes

Encrypted document support

Layer / File(s)	Summary
Transaction metadata cryptography packages/rs-platform-wallet/src/wallet/identity/crypto/..., packages/rs-platform-wallet/tests/legacy_wire_compat/*	Adds HD key derivation, AES-256-CBC sealing and opening, version and payload validation, redacted metadata types, and legacy compatibility checks.
Encrypted document wallet operations packages/rs-platform-wallet/src/wallet/identity/network/..., packages/rs-platform-wallet/tests/txmetadata_fetch.rs	Adds encrypted document preparation, decryption, pagination, cursor handling, exports, and a Testnet integration test.
Native FFI and runtime handling packages/rs-platform-wallet-ffi/src/*, packages/rs-platform-wallet-ffi/Cargo.toml	Adds encrypted document FFI operations, base64 JSON payloads, sensitive-data cleanup, typed validation mapping, and fallible worker execution.
JNI and Kotlin integration packages/rs-unified-sdk-jni/src/*, packages/kotlin-sdk/sdk/src/main/kotlin/...	Adds encrypted document JNI methods, argument validation, JSON conversion, Kotlin transaction methods, and bounded error breadcrumbs.
Typed Kotlin error mapping packages/kotlin-sdk/sdk/src/main/kotlin/org/dashfoundation/dashsdk/errors/*, packages/kotlin-sdk/sdk/src/test/kotlin/...	Adds PlatformWallet.InvalidParameter and tests its native-code, inheritance, and message behavior.

Estimated code review effort: 5 (Critical) | ~120 minutes

Sequence Diagram(s)

sequenceDiagram
  participant KotlinClient
  participant TransactionsNative
  participant JNITransactions
  participant PlatformWalletFFI
  participant IdentityWallet
  KotlinClient->>TransactionsNative: createEncryptedDocument or fetchEncryptedDocuments
  TransactionsNative->>JNITransactions: invoke encrypted JNI method
  JNITransactions->>PlatformWalletFFI: validate and call encrypted document FFI
  PlatformWalletFFI->>IdentityWallet: create or query encrypted documents
  IdentityWallet-->>PlatformWalletFFI: encrypted or decrypted document data
  PlatformWalletFFI-->>JNITransactions: JSON result
  JNITransactions-->>TransactionsNative: canonical JSON
  TransactionsNative-->>KotlinClient: document JSON

Loading

Possibly related issues

• [kotlin-sdk] Encrypted-document create/update for the wallet txMetadata contract #4086 — Covers Kotlin SDK encrypted-document creation and native encryption-key handling.

Possibly related PRs

• dashpay/platform#4243 — Overlaps in encrypted txMetadata document fetching, decryption, FFI serialization, and cleanup.

Suggested reviewers: quantumexplorer, lklimek

🚥 Pre-merge checks | ✅ 5

✅ Passed checks (5 passed)

Check name	Status	Explanation
Description Check	✅ Passed	Check skipped - CodeRabbit’s high-level summary is enabled.
Title check	✅ Passed	The title clearly and concisely describes the PR's main change: hardened encrypted txMetadata document support in the SDK.
Docstring Coverage	✅ Passed	Docstring coverage is 100.00% which is sufficient. The required threshold is 80.00%.
Linked Issues check	✅ Passed	Check skipped because no linked issues were found for this pull request.
Out of Scope Changes check	✅ Passed	Check skipped because no linked issues were found for this pull request.

✨ Finishing Touches 💡 1

🛠️ Fix failing CI checks 💡

• Create stacked PR
• Commit on current branch

📝 Generate docstrings

• Create stacked PR
• Commit on current branch

🧪 Generate unit tests (beta)

• Create PR with unit tests
• Commit unit tests in branch fix/txmetadata-encrypted-document-foundation

---

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

• X
• Mastodon
• Reddit
• LinkedIn

_{Comment @coderabbitai help to get the list of available commands.}

[thepastaclaw]

⛔ Blockers found — Sonnet deferred (commit bf88c92)
Canonical validated blockers: 1

[coderabbitai]

Actionable comments posted: 1

🧹 Nitpick comments (2)

packages/rs-platform-wallet/src/wallet/identity/crypto/tx_metadata.rs (1)

118-122: 📐 Maintainability & Code Quality | 🔵 Trivial | 💤 Low value

Remove the review-conversation references from the permanent doc.

Lines 121-122 and the test doc at lines 477-481 cite "the reviewer's figure" and "the review". A reader of the merged code has no access to that conversation. State the arithmetic without referring to the review thread.

📝 Proposed doc edit

 /// Note the envelope for the maximum plaintext is 4081 bytes, not 4096: the
 /// gap 4082..=4096 is unreachable because the next plaintext byte (4064) forces
-/// a fresh padding block that jumps straight to 4097. (The 4063/4064 boundary
-/// itself matches the reviewer's figure; the "4063 → 4096" envelope size in the
-/// review does not — see the module tests, which pin the real 4081-byte blob.)
+/// a fresh padding block that jumps straight to 4097. The module tests pin the
+/// real 4081-byte blob.

🤖 Prompt for AI Agents

Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

In `@packages/rs-platform-wallet/src/wallet/identity/crypto/tx_metadata.rs` around
lines 118 - 122, Update the documentation comment near the maximum plaintext
envelope and the related test documentation to remove references to the
reviewer, review, or review conversation. Preserve the arithmetic explanation
and the stated 4081-byte maximum, expressing the 4063/4064 boundary and
unreachable 4082..=4096 range as standalone technical facts.

packages/rs-unified-sdk-jni/src/support.rs (1)

101-116: 📐 Maintainability & Code Quality | 🔵 Trivial | 💤 Low value

Consider pub(crate) for the breadcrumb helpers.

platform_wallet_error_breadcrumb and thrown_exception_breadcrumb are internal diagnostic seams. Only this module and its tests call them. pub adds them to the crate's public surface, and the tests reach private items through super:: already, as net_from_ord shows at Line 184.

♻️ Proposed visibility narrowing

-pub fn platform_wallet_error_breadcrumb(platform_wallet_code: i32, _message: &str) -> String {
+pub(crate) fn platform_wallet_error_breadcrumb(platform_wallet_code: i32, _message: &str) -> String {

-pub fn thrown_exception_breadcrumb(code: i32, _message: &str) -> String {
+pub(crate) fn thrown_exception_breadcrumb(code: i32, _message: &str) -> String {

🤖 Prompt for AI Agents

Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

In `@packages/rs-unified-sdk-jni/src/support.rs` around lines 101 - 116, Change
the visibility of platform_wallet_error_breadcrumb and
thrown_exception_breadcrumb from pub to pub(crate), keeping their signatures and
behavior unchanged so they remain accessible within the crate and existing
tests.

🤖 Prompt for all review comments with AI agents

Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

Inline comments:
In
`@packages/rs-platform-wallet/src/wallet/identity/network/encrypted_document.rs`:
- Around line 597-639: Bound the pagination loop in the document-fetching flow
around Document::fetch_many by tracking the number of pages and enforcing a
maximum page limit, while also verifying that the next StartAfter cursor
advances beyond the previous cursor. Stop or return an appropriate error when
either guard detects no progress or the page limit is reached, preventing
repeated full pages from growing raw_docs indefinitely.

---

Nitpick comments:
In `@packages/rs-platform-wallet/src/wallet/identity/crypto/tx_metadata.rs`:
- Around line 118-122: Update the documentation comment near the maximum
plaintext envelope and the related test documentation to remove references to
the reviewer, review, or review conversation. Preserve the arithmetic
explanation and the stated 4081-byte maximum, expressing the 4063/4064 boundary
and unreachable 4082..=4096 range as standalone technical facts.

In `@packages/rs-unified-sdk-jni/src/support.rs`:
- Around line 101-116: Change the visibility of platform_wallet_error_breadcrumb
and thrown_exception_breadcrumb from pub to pub(crate), keeping their signatures
and behavior unchanged so they remain accessible within the crate and existing
tests.

🪄 Autofix (Beta)

Fix all unresolved CodeRabbit comments on this PR:

• Push a commit to this branch (recommended)
• Create a new PR with the fixes

---

ℹ️ Review info

⚙️ Run configuration

Configuration used: Path: .coderabbit.yaml

Review profile: CHILL

Plan: Pro Plus

Run ID: 48d089a2-0ca2-415f-9837-b19462d17093

📥 Commits

Reviewing files that changed from the base of the PR and between ed4116b and bf88c92.

⛔ Files ignored due to path filters (1)

• Cargo.lock is excluded by !**/*.lock

📒 Files selected for processing (22)

• packages/kotlin-sdk/sdk/src/main/kotlin/org/dashfoundation/dashsdk/documents/DocumentTransactions.kt
• packages/kotlin-sdk/sdk/src/main/kotlin/org/dashfoundation/dashsdk/errors/DashSdkError.kt
• packages/kotlin-sdk/sdk/src/main/kotlin/org/dashfoundation/dashsdk/ffi/TransactionsNative.kt
• packages/kotlin-sdk/sdk/src/test/kotlin/org/dashfoundation/dashsdk/errors/DashSdkErrorTest.kt
• packages/rs-platform-wallet-ffi/Cargo.toml
• packages/rs-platform-wallet-ffi/src/document.rs
• packages/rs-platform-wallet-ffi/src/error.rs
• packages/rs-platform-wallet-ffi/src/lib.rs
• packages/rs-platform-wallet-ffi/src/runtime.rs
• packages/rs-platform-wallet/Cargo.toml
• packages/rs-platform-wallet/src/error.rs
• packages/rs-platform-wallet/src/lib.rs
• packages/rs-platform-wallet/src/wallet/identity/crypto/mod.rs
• packages/rs-platform-wallet/src/wallet/identity/crypto/tx_metadata.rs
• packages/rs-platform-wallet/src/wallet/identity/network/encrypted_document.rs
• packages/rs-platform-wallet/src/wallet/identity/network/mod.rs
• packages/rs-platform-wallet/tests/legacy_wire_compat/LegacyDerivationPathCheck.java
• packages/rs-platform-wallet/tests/legacy_wire_compat/LegacyKeyN.java
• packages/rs-platform-wallet/tests/legacy_wire_compat/README.md
• packages/rs-platform-wallet/tests/txmetadata_fetch.rs
• packages/rs-unified-sdk-jni/src/support.rs
• packages/rs-unified-sdk-jni/src/transactions.rs

[coderabbitai]

🩺 Stability & Availability | 🟠 Major | ⚡ Quick win

Bound the pagination loop.

The loop continues while a page returns exactly PAGE entries and its last key is Some. The cursor advances only if the server honors StartAfter. If a node returns the same full page repeatedly, the loop never ends and raw_docs grows without limit, which hangs the call and exhausts memory.

Add a progress check on the cursor and a maximum page count.

🛡️ Proposed guard

     const PAGE: u32 = 100;
+    // A node that ignores `StartAfter` would otherwise spin forever and grow
+    // `raw_docs` without limit.
+    const MAX_PAGES: usize = 1_000;
     breadcrumb(&format!(
         "query_owned_encrypted_documents: entry since_ms={since_ms}"
     ));
     let mut raw_docs: Vec<(Identifier, Option<Document>)> = Vec::new();
     let mut start: Option<Start> = None;
+    let mut pages = 0usize;
     loop {
+        pages += 1;
+        if pages > MAX_PAGES {
+            breadcrumb_error(
+                "query_owned_encrypted_documents: page limit reached; stopping the scan",
+            );
+            break;
+        }

         match last_id {
-            Some(id) => start = Some(Start::StartAfter(id.to_buffer().to_vec())),
+            Some(id) => {
+                let next = Start::StartAfter(id.to_buffer().to_vec());
+                if Some(&next) == start.as_ref() {
+                    breadcrumb_error(
+                        "query_owned_encrypted_documents: cursor did not advance; stopping the scan",
+                    );
+                    break;
+                }
+                start = Some(next);
+            }
             None => break,
         }

🤖 Prompt for AI Agents

Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

In
`@packages/rs-platform-wallet/src/wallet/identity/network/encrypted_document.rs`
around lines 597 - 639, Bound the pagination loop in the document-fetching flow
around Document::fetch_many by tracking the number of pages and enforcing a
maximum page limit, while also verifying that the next StartAfter cursor
advances beyond the previous cursor. Stop or return an appropriate error when
either guard detects no progress or the page limit is reached, preventing
repeated full pages from growing raw_docs indefinitely.

[thepastaclaw]

Preliminary review — Codex only

The txMetadata foundation is well layered and its core cryptographic compatibility is extensively tested, but the C ABI still lets a non-derivable high-bit encryption key index copy plaintext and invoke the host mnemonic resolver before failing. Three non-blocking issues also remain around read-side version validation, secret residency during network fetches, and missing successful end-to-end fetch/decrypt coverage.

Validated blockers were found in the Codex precheck. Sonnet is deferred until a fresh Codex revalidation clears the blocker gate.

Review provenance

• Codex reviewers: gpt-5.6-sol — general (completed), gpt-5.6-sol — security-auditor (completed), gpt-5.6-sol — rust-quality (completed), gpt-5.6-sol — ffi-engineer (completed)
• Verifier: gpt-5.6-sol — verifier
• Sonnet: not run (deferred by blocker gate)

🔴 1 blocking | 🟡 3 suggestion(s)

🤖 Prompt for all review comments with AI agents

These findings are from an automated code review. Verify each finding against the current code and only fix it if needed.

In `packages/rs-platform-wallet-ffi/src/document.rs`:
- [BLOCKING] packages/rs-platform-wallet-ffi/src/document.rs:474-478: Reject non-derivable encryption key indices before resolver access
  The C ABI accepts every `u32` for `encryption_key_index`, but derivation eventually passes it to `ChildNumber::from_hardened_idx`, which rejects indices with bit 31 set. A request such as `0x8000_0000` therefore cannot succeed, yet this boundary copies the plaintext at lines 490-495 and, for an external-signable wallet, invokes the host mnemonic resolver at lines 511-513 before derivation rejects it. This violates the early-validation boundary established for payload length and wire version and can trigger a user-visible keychain prompt for an impossible request. The resulting `InvalidIdentityData` also maps to `ErrorUnknown` rather than the typed invalid-parameter result. Add a shared core/FFI range check before payload materialization, wallet lookup, or resolver access, and cover `0x8000_0000` with a regression test.
- [SUGGESTION] packages/rs-platform-wallet-ffi/src/document.rs:665-701: Acquire txMetadata key material after network fetching
  The external-signable path resolves the mnemonic-derived master xprv before fetching the contract and all document pages, then retains it across every network await. The resident path similarly clones the key-bearing wallet before the document query. This unnecessarily prompts the resolver for missing contracts or empty results and lets a slow or stalled network extend secret residency indefinitely. The comment stating that derivation occurs between page fetches does not match the implementation: `query_owned_encrypted_documents` completes all pagination before the synchronous decrypt loop starts. Split raw document retrieval from decryption and acquire the resolver master or resident wallet only after the network phase returns materialized documents that require decryption.

In `packages/rs-platform-wallet/src/wallet/identity/crypto/tx_metadata.rs`:
- [SUGGESTION] packages/rs-platform-wallet/src/wallet/identity/crypto/tx_metadata.rs:379-380: Reject unsupported versions when opening txMetadata
  The create path declares versions 0 and 1 as the only supported wire formats, but `open_tx_metadata` accepts any leading byte. Because the version byte is outside the ciphertext and IV, changing a valid blob's first byte to 2 does not affect CBC decryption or padding, so the fetch path can return successfully decrypted data tagged with an unsupported version. Validate the version before decryption so malformed or unsupported documents follow the documented skip-on-error path instead of producing a result outside the API's supported domain.

In `packages/rs-platform-wallet/tests/txmetadata_fetch.rs`:
- [SUGGESTION] packages/rs-platform-wallet/tests/txmetadata_fetch.rs:15-17: Exercise the successful fetch-to-decrypt path in an automated test
  No automated test successfully calls `IdentityWallet::fetch_encrypted_documents`. The network-free tests exercise crypto directly, while this ignored test calls only `query_owned_encrypted_documents` and then independently reimplements field extraction. It therefore does not verify the production orchestration that maps document properties, derives per-document keys, skips malformed entries, propagates the version, or maps IDs and timestamps into `DecryptedEncryptedDocument`. Add a mock-SDK test with a managed wallet, one valid sealed document, and one malformed document, then drive the public fetch method and assert the complete valid result plus skip behavior.

[thepastaclaw]

🔴 Blocking: Reject non-derivable encryption key indices before resolver access

The C ABI accepts every u32 for encryption_key_index, but derivation eventually passes it to ChildNumber::from_hardened_idx, which rejects indices with bit 31 set. A request such as 0x8000_0000 therefore cannot succeed, yet this boundary copies the plaintext at lines 490-495 and, for an external-signable wallet, invokes the host mnemonic resolver at lines 511-513 before derivation rejects it. This violates the early-validation boundary established for payload length and wire version and can trigger a user-visible keychain prompt for an impossible request. The resulting InvalidIdentityData also maps to ErrorUnknown rather than the typed invalid-parameter result. Add a shared core/FFI range check before payload materialization, wallet lookup, or resolver access, and cover 0x8000_0000 with a regression test.

_{source: ['codex']}

[thepastaclaw]

🟡 Suggestion: Reject unsupported versions when opening txMetadata

The create path declares versions 0 and 1 as the only supported wire formats, but open_tx_metadata accepts any leading byte. Because the version byte is outside the ciphertext and IV, changing a valid blob's first byte to 2 does not affect CBC decryption or padding, so the fetch path can return successfully decrypted data tagged with an unsupported version. Validate the version before decryption so malformed or unsupported documents follow the documented skip-on-error path instead of producing a result outside the API's supported domain.

Suggested change

	let version = blob[0];
	let iv: [u8; 16] = blob[1..BLOB_HEADER_LEN]
	let version = blob[0];
	ensure_tx_metadata_version_supported(version)?;
	let iv: [u8; 16] = blob[1..BLOB_HEADER_LEN]

_{source: ['codex']}

[thepastaclaw]

🟡 Suggestion: Acquire txMetadata key material after network fetching

The external-signable path resolves the mnemonic-derived master xprv before fetching the contract and all document pages, then retains it across every network await. The resident path similarly clones the key-bearing wallet before the document query. This unnecessarily prompts the resolver for missing contracts or empty results and lets a slow or stalled network extend secret residency indefinitely. The comment stating that derivation occurs between page fetches does not match the implementation: query_owned_encrypted_documents completes all pagination before the synchronous decrypt loop starts. Split raw document retrieval from decryption and acquire the resolver master or resident wallet only after the network phase returns materialized documents that require decryption.

_{source: ['codex']}

[thepastaclaw]

🟡 Suggestion: Exercise the successful fetch-to-decrypt path in an automated test

No automated test successfully calls IdentityWallet::fetch_encrypted_documents. The network-free tests exercise crypto directly, while this ignored test calls only query_owned_encrypted_documents and then independently reimplements field extraction. It therefore does not verify the production orchestration that maps document properties, derives per-document keys, skips malformed entries, propagates the version, or maps IDs and timestamps into DecryptedEncryptedDocument. Add a mock-SDK test with a managed wallet, one valid sealed document, and one malformed document, then drive the public fetch method and assert the complete valid result plus skip behavior.

_{source: ['codex']}