Macro for constructing a high-level type-safe wrapper around substrate storage

[rphmeier]

Goal: never reference or load storage items using the key string directly. It is arcane, bug-prone, and unreadable.

usage:

Using a trait Storage:

trait Storage {
    // panic if the type is wrong for the key.
    fn load<T: Decodable>(&self) -> Option<T>;
    fn store<T: Encodable>(&mut self, value: T);
}

storage_declarations! {
    Authorities: List(":auth" -> AuthorityId), // creates something like current `KeyedVec` using prefix ":auth"
    Code: ":code" -> Vec<u8>, // creates a single value. stored under that key.
    ...
}

Authorities::len_key() -> &'static [u8];
Authorities::load_len(&Storage) -> u32
Authorities::key_for(n) -> Vec<u8>;
Authorities::load_from(&Storage, n) -> Option<AuthorityId>;
// ... KeyedVec-like API

Code::load_from(&Storage) -> Option<Vec<u8>>; // assumes a `Decodable` trait where the `<[u8] as Decodable>::Decoded = Vec<u8>`
Code::store_in(&mut Storage, &[u8]); 
Code::key() -> &'static [u8]; // for low-level usage.

crate substrate_storage would define all storage values used in substrate.
crate polkadot_storage would define all storage values used in polkadot.

The "load"/"store" API is a little annoying, so under runtime-support we would provide a Storage implementation that calls out to the externalities and a trait to provide helpers that are more ergonomic: i.e. a load() and store(T) function which are usable only within the runtime.

Usage in runtime:

// assuming these declarations:
storage_declarations! {
    Authorities: List(":auth" -> AuthorityId), // creates something like current `KeyedVec` using prefix ":auth"
    Code: ":code" -> Vec<u8>, // creates a single value. stored under that key.
    ...
}

// ...

Authorities::len_key() -> &'static [u8];
Authorities::len() -> u32
Authorities::key_for(n) -> Vec<u8>;
Authorities::load(n) -> AuthorityId;
// ... KeyedVec-like API

Code::load() -> Option<Vec<u8>>; // assumes a `Decodable` trait where the `<[u8] as Decodable>::Decoded = Vec<u8>`
Code::store(&[u8]); 
Code::key() -> &'static [u8]; // for low-level usage.

[gavofyork]

crate substrate_storage would define all storage values used in substrate.

what is this fascination with collecting common lexicographical items into crates!?

while i agree with the general gist of this issue, storage elements should be declared (and exported as needed) in the modules that use them, not in some monolithic collection, just like constants, types, helper functions and tests.

[rphmeier]

@gavofyork sure, then those particular keys like :auth: and :code: should be defined in runtime-std as there isn't really anywhere else suitable.

[rphmeier]

In general the goal of extracting the constants out to a separate crate is to allow type-safe access of the storage without having to link to the entire runtime.