Extend node with build-test-clone subcommand by ales-otf · Pull Request #2489 · opentensor/subtensor

ales-otf · 2026-03-10T17:38:06Z

Description

Introduces build-test-clone subcommand, that does next:

syncs the state of the given network
exports it into a temporary chainspec
patches the chainspec to make a test network chainspec from it (applying this patch by @l0r1s )

So basically it produces a patched clone of a given network for testing. So you can test on mainnet clone, for example.

It can be used like this:

target/release/node-subtensor build-test-clone --base-path target/mainnet-clone --chain chainspecs/raw_spec_finney.json --bootnodes /dns/bootnode.finney.chain.opentensor.ai/tcp/30333/ws/p2p/12D3KooWRwbMb85RWnT8DSXSYMWQtuDwh4LJzndoRrTDotTR5gDC --output target/mainnet-clone-chainspec.json

Then you normally run a testing node with produced chainspec like this:

target/release/node-subtensor --base-path target/clone --chain target/mainnet-clone-chainspec.json --database paritydb --force-authoring --alice --validator --unsafe-force-node-key-generation

The PR introduces a history_backfill flag to skip the gap feeling phase after the initial sync. That phase is needed for the historical data, but for the testing purposes we can skip it. It's keep by default, so it doesn't affect current node behavior. It's skip by default for build-test-clone to speed up the process.

Also unlike normal node default to rocksdb, build-test-clone is default to paritydb. And unlike the patch, build-test-clone produces chainspec with only single authority (alice) by default. This is more convenient and lightweight for local runs. But you can specify other conventional authorities using flags (--alice, --bob, --charlie) - any combination of them are allowed.

❯ target/release/node-subtensor build-test-clone -h
Build a patched clone chainspec by syncing state, exporting raw state, and applying test patch

Usage: node-subtensor build-test-clone [OPTIONS] --chain <CHAIN> --base-path <PATH> --output <FILE>

Options:
      --chain <CHAIN>                        Chain spec identifier or path (same semantics as `--chain`)
      --base-path <PATH>                     Base path used for syncing and state export
      --output <FILE>                        Output file path for the final patched chainspec JSON
      --sync <SYNC>                          Sync mode for the temporary sync node [default: warp] [possible values: full, fast, fast-unsafe, warp]
      --database <DATABASE>                  Database backend for the temporary sync/export node [default: paritydb] [possible values: rocksdb, paritydb, auto, paritydb-experimental]
      --rpc-port <RPC_PORT>                  RPC port used by the temporary sync node [default: 9966]
      --port <PORT>                          P2P port used by the temporary sync node [default: 30466]
      --sync-timeout-sec <SYNC_TIMEOUT_SEC>  Maximum time to wait for sync completion [default: 7200]
      --sync-lag-blocks <SYNC_LAG_BLOCKS>    Accept sync completion when current is within this many blocks of highest [default: 8]
      --bootnodes <BOOTNODE>                 Optional bootnodes for the sync step. Repeatable
      --alice                                Include Alice in patched validator authorities (default if no validator flags are passed)
      --bob                                  Include Bob in patched validator authorities (if any validator flag is set, only selected validators are used)
      --charlie                              Include Charlie in patched validator authorities (if any validator flag is set, only selected validators are used)
  -h, --help                                 Print help (see more with '--help')

Related Issue(s)

Closes #[issue number]

Type of Change

Bug fix (non-breaking change which fixes an issue)
New feature (non-breaking change which adds functionality)
Breaking change (fix or feature that would cause existing functionality to not work as expected)
Documentation update
Other (please describe):

Breaking Change

If this PR introduces a breaking change, please provide a detailed description of the impact and the migration path for existing applications.

Checklist

I have performed a self-review of my own code
I have commented my code, particularly in hard-to-understand areas
I have run ./scripts/fix_rust.sh to ensure my code is formatted and linted correctly
I have made corresponding changes to the documentation
My changes generate no new warnings
I have added tests that prove my fix is effective or that my feature works
New and existing unit tests pass locally with my changes
Any dependent changes have been merged and published in downstream modules

Screenshots (if applicable)

Please include any relevant screenshots or GIFs that demonstrate the changes made.

Additional Notes

Please provide any additional information or context that may be helpful for reviewers.

l0r1s · 2026-03-10T22:16:58Z

node/src/cli.rs

    ChainInfo(sc_cli::ChainInfoCmd),
+
+    // Build a patched test clone chainspec from synced network state.
+    #[command(name = "build-test-clone")]


I'm not sure the name reflect what the code is doing because what it is really doing is building a chainspec, right?

Maybe something like build-clone-spec ?

oh, i was thinking about it a lot. initially it was clone-state and i also considered the one you're suggesting. i don't know, actually reflecting name would be something like clone-into-patched-spec, build-patched-clone and then build-test-clone or something like that. i'm not happy with any of these variants.

l0r1s · 2026-03-10T22:22:08Z

node/src/cli.rs

+    /// Chain spec identifier or path (same semantics as `--chain`).
+    #[arg(long, value_name = "CHAIN")]
+    pub chain: String,
+
+    /// Base path used for syncing and state export.
+    #[arg(long, value_name = "PATH")]
+    pub base_path: PathBuf,


Maybe we can have a default for those? We will use it for mainnet most of the time and using target/mainnet-clone seems like a good default?

this is a good default if we consider it a source code/cargo environment, but node is a binary though and can be ran everywhere. i would keep it explicit, because even if we use something like tmp, it's easy to miss and clutter up your storage.

l0r1s · 2026-03-10T22:34:27Z

node/src/clone_spec.rs

+            "--history-backfill",
+            history_backfill.as_ref(),


This caused a panics at the end of the sync because it doesn't exists on export-state, need to be kept only for the main command I guess:

2026-03-10 19:28:10 build-test-clone: sync target reached 2026-03-10 19:28:12 Essential task `txpool-background` failed. Shutting down service. error: unexpected argument '--history-backfill' found tip: to pass '--history-backfill' as a value, use '-- --history-backfill' Usage: node-subtensor export-state --chain <CHAIN_SPEC> --base-path <PATH> --database <DB> [HASH or NUMBER] For more information, try '--help'. 2026-03-10 19:28:13 build-test-clone: exporting raw state Error: Application("export-state failed with status exit status: 2")

oh shit, i broke it with my clean up 😄

l0r1s · 2026-03-10T22:40:08Z

node/src/cli.rs

+    #[arg(long, default_value_t = false)]
+    pub alice: bool,
+
+    /// Include Bob in patched validator authorities (if any validator flag is set, only selected validators are used).
+    #[arg(long, default_value_t = false)]
+    pub bob: bool,
+
+    /// Include Charlie in patched validator authorities (if any validator flag is set, only selected validators are used).
+    #[arg(long, default_value_t = false)]
+    pub charlie: bool,


What could be nice is to have an "authorities" vector arg so we can add more than 3 authorities if needed and not fixed ones (alice/bob/charlie), else we have to repatch the chainspec after creating it. Maybe it can be done later if we need it though!

i'd keep alice, bob and charlie as flags as we have with node and use when we run node for testing, but having additional flag to specify other authorities would be useful though.

l0r1s · 2026-03-10T22:44:24Z

node/src/clone_spec.rs

+        "--no-telemetry".to_string(),
+        "--no-prometheus".to_string(),
+        "--no-mdns".to_string(),
+        "--name".to_string(),


Missing something after the --name, or maybe it's not needed?

the name is on the next line :) it's build-test-clone-sync.

l0r1s · 2026-03-10T22:45:51Z

node/src/clone_spec.rs

+static VALIDATORS: &[Validator] = &[
+    Validator {
+        name: "alice",
+        sr25519_hex: "d43593c715fdd31c61141abd04a99fd6822c8558854ccde39a5684e7a56da27d",
+        ed25519_hex: "88dc3417d5058ec4b4503e0c12ea1a0a89be200fe98922423d4334014fa6b0ee",
+    },
+    Validator {
+        name: "bob",
+        sr25519_hex: "8eaf04151687736326c9fea17e25fc5287613693c912909cb226aa4794f26a48",
+        ed25519_hex: "d17c2d7823ebf260fd138f2d7e27d114c0145d968b5ff5006125f2414fadae69",
+    },
+    Validator {
+        name: "charlie",
+        sr25519_hex: "90b5ab205c6974c9ea841be688864633dc9ca8a357843eeacf2314649965fe22",
+        ed25519_hex: "439660b36c6c03afafca027b910b4fecf99801834c62a5e6006f27d978de234f",
+    },
+];


There are 2 functions to derive this from the name in the node/chainspec/mod.rs to avoid hardcoding?

i'll take a look, thanks!

ales-otf added 3 commits March 10, 2026 17:44

Add build-test-clone subcommand

f34447f

Add tests

0652bcc

Merge branch 'devnet-ready' into feature/build-test-clone

c14c81d

ales-otf added the skip-cargo-audit This PR fails cargo audit but needs to be merged anyway label Mar 10, 2026

ales-otf force-pushed the feature/build-test-clone branch from c0340ee to 71346f1 Compare March 10, 2026 19:22

Clean up

b9b0e7a

ales-otf force-pushed the feature/build-test-clone branch from 71346f1 to b9b0e7a Compare March 10, 2026 19:43

ales-otf marked this pull request as ready for review March 10, 2026 20:19

ales-otf requested review from JohnReedV, gztensor, l0r1s, open-junius, sam0x17 and shamil-gadelshin March 10, 2026 20:19

l0r1s reviewed Mar 10, 2026

View reviewed changes

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Extend node with build-test-clone subcommand#2489

Extend node with build-test-clone subcommand#2489
ales-otf wants to merge 4 commits intodevnet-readyfrom
feature/build-test-clone

ales-otf commented Mar 10, 2026 •

edited

Loading

Uh oh!

l0r1s Mar 10, 2026

Uh oh!

ales-otf Mar 10, 2026

Uh oh!

l0r1s Mar 10, 2026 •

edited

Loading

Uh oh!

ales-otf Mar 10, 2026

Uh oh!

l0r1s Mar 10, 2026

Uh oh!

ales-otf Mar 10, 2026

Uh oh!

l0r1s Mar 10, 2026 •

edited

Loading

Uh oh!

ales-otf Mar 10, 2026

Uh oh!

l0r1s Mar 10, 2026

Uh oh!

ales-otf Mar 10, 2026

Uh oh!

l0r1s Mar 10, 2026 •

edited

Loading

Uh oh!

ales-otf Mar 10, 2026

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

2 participants

Conversation

ales-otf commented Mar 10, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Description

Related Issue(s)

Type of Change

Breaking Change

Checklist

Screenshots (if applicable)

Additional Notes

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

l0r1s Mar 10, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

l0r1s Mar 10, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

l0r1s Mar 10, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

2 participants

ales-otf commented Mar 10, 2026 •

edited

Loading

l0r1s Mar 10, 2026 •

edited

Loading

l0r1s Mar 10, 2026 •

edited

Loading

l0r1s Mar 10, 2026 •

edited

Loading