Skip to content

refactor: prefill RPC cache for collect_events#7077

Merged
hanabi1224 merged 18 commits into
mainfrom
hm/prefill-cache-collect-events
May 21, 2026
Merged

refactor: prefill RPC cache for collect_events#7077
hanabi1224 merged 18 commits into
mainfrom
hm/prefill-cache-collect-events

Conversation

@hanabi1224
Copy link
Copy Markdown
Contributor

@hanabi1224 hanabi1224 commented May 18, 2026

Summary of changes

Changes introduced in this pull request:

  • prefill RPC cache for collect_events
  • remove cacheless resolve_to_key_addr in favor of id_to_deterministic_address. A few RPC methods (Filecoin.GasEstimateGasLimit, Filecoin.MpoolPushMessage, eth_call, trace_call) should benefit from this.
  • add automatic cache hit/miss metrics for every SizeTrackingCache
  • update docs accordingly

id_to_deterministic_address cache has a pretty good hit rate

# HELP cache_id_to_deterministic_address_hits Cache hits of id_to_deterministic_address
# TYPE cache_id_to_deterministic_address_hits gauge
cache_id_to_deterministic_address_hits 17708
# HELP cache_id_to_deterministic_address_misses Cache misses of id_to_deterministic_address
# TYPE cache_id_to_deterministic_address_misses gauge
cache_id_to_deterministic_address_misses 2074

Reference issue to close (if applicable)

Closes

Other information and links

Change checklist

  • I have performed a self-review of my own code,
  • I have made corresponding changes to the documentation. All new code adheres to the team's documentation standards,
  • I have added tests that prove my fix is effective or that my feature works (if possible),
  • I have made sure the CHANGELOG is up-to-date. All user-facing changes should be reflected in this document.

Outside contributions

  • I have read and agree to the CONTRIBUTING document.
  • I have read and agree to the AI Policy document. I understand that failure to comply with the guidelines will lead to rejection of the pull request.

Summary by CodeRabbit

  • New Features

    • Events are now collected and cached during RPC cache warmup for newly validated tipsets.
  • Refactor

    • Unified caching with new sync/async get-or-insert semantics and direct size-tracking cache usage.
    • Standardized deterministic address resolution across chain sync, RPC, wallet, message-pool, interpreter, and state-manager paths.
    • Increased default cache sizes and simplified cache/shallow-clone ergonomics.
  • Chores

    • Updated dependency declarations and reorganized imports for consistency.

Review Change Stack

@hanabi1224 hanabi1224 added the RPC requires calibnet RPC checks to run on CI label May 18, 2026
@coderabbitai
Copy link
Copy Markdown
Contributor

coderabbitai Bot commented May 18, 2026

Note

Reviews paused

It looks like this branch is under active development. To avoid overwhelming you with review comments due to an influx of new commits, CodeRabbit has automatically paused this review. You can configure this behavior by changing the reviews.auto_review.auto_pause_after_reviewed_commits setting.

Use the following commands to manage reviews:

  • @coderabbitai resume to resume automatic reviews.
  • @coderabbitai review to trigger a single review.

Use the checkboxes below for quick actions:

  • ▶️ Resume reviews
  • 🔍 Trigger review

Walkthrough

Adds sync/async get-or-insert APIs to SizeTrackingCache, promotes quick_cache dependency, standardizes CidWrapper and prelude usage, migrates all callers from resolve_to_key_addr to resolve_to_deterministic_address, refactors several cache wrappers to hold SizeTrackingCache directly, and updates ETH RPC caching and event-collection call sites.

Changes

Cache and Address Resolution Consolidation

Layer / File(s) Summary
SizeTrackingCache APIs & deps
Cargo.toml, src/utils/cache/size_tracking.rs
Adds get_or_insert_with and get_or_insert_async (and inline annotations) to SizeTrackingCache; moves quick_cache = "0.6" to main dependencies and bumps get-size2 to 0.9.
Prelude & import consolidation
src/lib.rs, src/db/car/forest.rs, src/db/car/mod.rs, src/db/blockstore_with_read_cache.rs, src/message_pool/msgpool/selection.rs, src/message_pool/msgpool/msg_pool.rs
Re-exports CidWrapper via crate prelude and converts modules to use crate::prelude::* for common types.
CidWrapper equivalence & block caches
src/utils/get_size/mod.rs, src/chain_sync/bad_block_cache.rs, src/db/blockstore_with_read_cache.rs, src/db/car/forest.rs
Implements quick_cache::Equivalent<CidWrapper> for Cid; standardizes cache key types to CidWrapper and updates lookups/insertions accordingly.
Cache wrapper refactors
src/chain/store/chain_store.rs, src/state_migration/common/mod.rs, src/state_manager/cache.rs
Converts MessagesInTipsetCache, MigrationCache, and ForestCache to tuple structs containing SizeTrackingCache; updates API shapes, shallow_clone, and tests accordingly.
MigrationCache & StateMigration usage
src/state_migration/*
Change MigrationCache API to accept &str keys, add ShallowClone, and update StateMigration and MinerMigrator to use shallow_clone() and borrowed keys.
StateTree & StateManager address resolution
src/shim/state_tree.rs, src/state_manager/address_resolution.rs, src/state_manager/cache.rs, src/state_manager/mod.rs
Rename resolve_to_deterministic_addrresolve_to_deterministic_address, prefer delegated_address when present, remove legacy resolve_to_key_addr helper, update get_bls_public_key signature to take Address by value, and use async cache population for deterministic-address resolution.
Remove legacy helper & update callers
src/interpreter/mod.rs, src/interpreter/fvm2.rs, src/interpreter/fvm3.rs, src/interpreter/fvm4.rs, src/rpc/methods/gas.rs, src/rpc/methods/mpool.rs, src/rpc/methods/wallet.rs, src/message_pool/msgpool/provider.rs, src/state_manager/message_simulation.rs
Remove free resolve_to_key_addr helper and update callers to call state.resolve_to_deterministic_address(...) or state_manager.resolve_to_deterministic_address(...).
ETH RPC caching & event API
src/rpc/methods/eth.rs, src/rpc/methods/eth/filter/mod.rs, src/daemon/mod.rs
Switch ETH block caches from ForestCache to SizeTrackingCache using get_or_insert_async; change EthEventHandler::collect_events to accept &StateManager and update call sites; daemon warmup pre-fills events.
Message-pool signature cache & utils
src/message_pool/msgpool/msg_pool.rs, src/message_pool/msgpool/utils.rs
Use Cid/CidWrapper::from directly for signature-validation and BLS caches (get_cloned/push), and adjust small imports.
Minor import/test adjustments
tests/*, various small files
Adjust tests and small import rearrangements to match API and import changes (Arc/Arc vs Vec, .0 access, import ordering).

Sequence Diagram

sequenceDiagram
  participant RPC as RPC / Daemon
  participant EthHandler as EthEventHandler
  participant StateMgr as StateManager
  participant Cache as SizeTrackingCache
  RPC->>EthHandler: collect_events(&StateMgr, tipset)
  EthHandler->>StateMgr: load_executed_tipset(...)
  EthHandler->>StateMgr: resolve_to_deterministic_address(...)
  EthHandler->>Cache: get_or_insert_async(key, compute)
  Cache-->>EthHandler: cached events / newly computed events
  EthHandler-->>RPC: return events (or warn on error)
Loading

Estimated code review effort

🎯 4 (Complex) | ⏱️ ~45 minutes

Possibly related PRs

Suggested reviewers

  • LesnyRumcajs
  • akaladarshi
🚥 Pre-merge checks | ✅ 5
✅ Passed checks (5 passed)
Check name Status Explanation
Title check ✅ Passed The title 'refactor: prefill RPC cache for collect_events' is directly related to the PR's main objective of prefilling the RPC cache for event collection, as evidenced by changes in src/daemon/mod.rs (event cache prefilling) and src/rpc/methods/eth.rs (cache refactoring).
Docstring Coverage ✅ Passed Docstring coverage is 86.21% which is sufficient. The required threshold is 80.00%.
Linked Issues check ✅ Passed Check skipped because no linked issues were found for this pull request.
Out of Scope Changes check ✅ Passed Check skipped because no linked issues were found for this pull request.
Description Check ✅ Passed Check skipped - CodeRabbit’s high-level summary is enabled.

✏️ Tip: You can configure your own custom pre-merge checks in the settings.

✨ Finishing Touches
🧪 Generate unit tests (beta)
  • Create PR with unit tests
  • Commit unit tests in branch hm/prefill-cache-collect-events
✨ Simplify code
  • Create PR with simplified code
  • Commit simplified code in branch hm/prefill-cache-collect-events

Comment @coderabbitai help to get the list of available commands and usage tips.

@hanabi1224 hanabi1224 marked this pull request as ready for review May 18, 2026 21:37
@hanabi1224 hanabi1224 requested a review from a team as a code owner May 18, 2026 21:37
@hanabi1224 hanabi1224 requested review from LesnyRumcajs and akaladarshi and removed request for a team May 18, 2026 21:37
Copy link
Copy Markdown
Contributor

@coderabbitai coderabbitai Bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Actionable comments posted: 1

🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

Inline comments:
In `@src/state_manager/address_resolution.rs`:
- Around line 61-82: The initializer passed to
id_to_deterministic_address_cache.get_or_insert_async is performing an extra
insertion by calling self.id_to_deterministic_address_cache.push(id, resolved);
remove that redundant push so the initializer only computes and returns
Ok(resolved) (letting SizeTrackingCache::get_or_insert_async handle
insertion/size accounting); you can keep the resolution logic
(StateTree::new_from_root, resolve_to_deterministic_address, load_tipset_state)
unchanged—just delete the push call inside the async block.
🪄 Autofix (Beta)

Fix all unresolved CodeRabbit comments on this PR:

  • Push a commit to this branch (recommended)
  • Create a new PR with the fixes

ℹ️ Review info
⚙️ Run configuration

Configuration used: Repository UI

Review profile: CHILL

Plan: Pro

Run ID: 39d054a6-e444-407d-9bc5-765dfa106e52

📥 Commits

Reviewing files that changed from the base of the PR and between e0bca2a and b869240.

⛔ Files ignored due to path filters (1)
  • Cargo.lock is excluded by !**/*.lock
📒 Files selected for processing (30)
  • Cargo.toml
  • src/chain/store/chain_store.rs
  • src/chain_sync/bad_block_cache.rs
  • src/chain_sync/tipset_syncer.rs
  • src/daemon/mod.rs
  • src/db/blockstore_with_read_cache.rs
  • src/db/car/forest.rs
  • src/db/car/mod.rs
  • src/interpreter/fvm2.rs
  • src/interpreter/fvm3.rs
  • src/interpreter/fvm4.rs
  • src/interpreter/mod.rs
  • src/lib.rs
  • src/message_pool/msgpool/msg_pool.rs
  • src/message_pool/msgpool/provider.rs
  • src/message_pool/msgpool/selection.rs
  • src/rpc/methods/eth.rs
  • src/rpc/methods/eth/filter/mod.rs
  • src/rpc/methods/gas.rs
  • src/rpc/methods/mpool.rs
  • src/rpc/methods/wallet.rs
  • src/shim/state_tree.rs
  • src/state_manager/address_resolution.rs
  • src/state_manager/cache.rs
  • src/state_manager/message_simulation.rs
  • src/state_manager/mod.rs
  • src/state_migration/common/mod.rs
  • src/state_migration/common/state_migration.rs
  • src/state_migration/nv21/miner.rs
  • src/utils/cache/size_tracking.rs
💤 Files with no reviewable changes (2)
  • src/db/car/mod.rs
  • src/interpreter/mod.rs

Comment thread src/state_manager/address_resolution.rs
@hanabi1224 hanabi1224 force-pushed the hm/prefill-cache-collect-events branch from b869240 to 45a4eff Compare May 18, 2026 21:54
@hanabi1224 hanabi1224 force-pushed the hm/prefill-cache-collect-events branch from 45a4eff to a8404ef Compare May 18, 2026 22:05
@codecov
Copy link
Copy Markdown

codecov Bot commented May 18, 2026

Codecov Report

❌ Patch coverage is 50.00000% with 115 lines in your changes missing coverage. Please review.
✅ Project coverage is 64.15%. Comparing base (3b7dcd7) to head (92f0620).
✅ All tests successful. No failed tests found.

Files with missing lines Patch % Lines
src/daemon/mod.rs 0.00% 44 Missing ⚠️
src/utils/cache/size_tracking.rs 6.45% 29 Missing ⚠️
src/rpc/methods/eth/filter/mod.rs 14.28% 16 Missing and 2 partials ⚠️
src/state_manager/address_resolution.rs 80.00% 1 Missing and 3 partials ⚠️
src/chain/store/index.rs 81.81% 2 Missing ⚠️
src/beacon/drand.rs 83.33% 1 Missing ⚠️
src/beacon/signatures/public_key_impls.rs 0.00% 1 Missing ⚠️
src/beacon/signatures/signature_impls.rs 0.00% 1 Missing ⚠️
src/chain_sync/tipset_syncer.rs 50.00% 0 Missing and 1 partial ⚠️
src/chain_sync/validation.rs 50.00% 1 Missing ⚠️
... and 13 more
Additional details and impacted files
Files with missing lines Coverage Δ
src/beacon/signatures/mod.rs 81.92% <100.00%> (ø)
src/blocks/tipset.rs 85.97% <ø> (+0.75%) ⬆️
src/chain/store/chain_store.rs 68.87% <100.00%> (-1.07%) ⬇️
src/chain_sync/bad_block_cache.rs 84.84% <100.00%> (+5.36%) ⬆️
src/cid_collections/hash_set.rs 77.08% <100.00%> (+5.78%) ⬆️
src/cid_collections/mod.rs 97.14% <ø> (-0.12%) ⬇️
src/db/blockstore_with_read_cache.rs 93.33% <100.00%> (ø)
src/db/car/forest.rs 83.87% <100.00%> (ø)
src/libp2p_bitswap/store.rs 77.77% <ø> (+7.77%) ⬆️
src/message_pool/mpool_locker.rs 100.00% <100.00%> (ø)
... and 35 more

... and 7 files with indirect coverage changes


Continue to review full report in Codecov by Sentry.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update 3b7dcd7...92f0620. Read the comment docs.

🚀 New features to boost your workflow:
  • 📦 JS Bundle Analysis: Save yourself from yourself by tracking and limiting bundle sizes in JS merges.

Comment thread src/state_manager/mod.rs
Comment thread src/state_manager/mod.rs
Comment thread src/chain/store/chain_store.rs
Comment thread src/state_manager/address_resolution.rs
Copy link
Copy Markdown
Collaborator

@akaladarshi akaladarshi left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@hanabi1224 There are some documents where we are still haven't updated the lru cache metrics such as metrics.md. Please update those as well.

Comment thread src/chain_sync/bad_block_cache.rs Outdated
Comment thread src/utils/cache/size_tracking.rs Outdated
Comment thread src/rpc/methods/eth.rs
Comment thread src/daemon/mod.rs Outdated
Copy link
Copy Markdown
Contributor

@coderabbitai coderabbitai Bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Actionable comments posted: 1

Caution

Some comments are outside the diff and can’t be posted inline due to platform limitations.

⚠️ Outside diff range comments (1)
src/state_manager/cache.rs (1)

62-74: ⚠️ Potential issue | 🟡 Minor | ⚡ Quick win

Hit/miss accounting is reversed here.

Line 66 only executes when the insert future is actually polled, so it marks a cache miss, not a hit. As written, computed entries increment TIPSET_HIT, while cached reads increment TIPSET_MISS.

Suggested fix
-        let mut hit = false;
+        let mut miss = false;
         let value = self
             .cache
             .get_or_insert_async(key, async {
-                hit = true;
+                miss = true;
                 compute().await
             })
             .await?;
-        if hit {
-            TIPSET_HIT.inc();
-        } else {
-            TIPSET_MISS.inc();
-        }
+        if miss {
+            TIPSET_MISS.inc();
+        } else {
+            TIPSET_HIT.inc();
+        }
🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

In `@src/state_manager/cache.rs` around lines 62 - 74, The hit/miss flag is
inverted because the async closure given to get_or_insert_async runs only on
cache misses; change the logic so hit starts true and the closure sets hit =
false (so a computed entry marks a miss), then keep the TIPSET_HIT / TIPSET_MISS
increments as-is; update the mutable variable usage around get_or_insert_async
and the async closure that calls compute() to flip the boolean assignment (use
hit = true before calling, and inside the async closure set hit = false).
🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

Inline comments:
In `@src/utils/cache/size_tracking.rs`:
- Around line 187-195: The compilation fails because get_or_insert_async uses
impl Future but std::future::Future is not imported; add the import "use
std::future::Future;" to the module's imports (top of
src/utils/cache/size_tracking.rs) so the function signature in
get_or_insert_async can resolve the Future trait; ensure the new use is grouped
with other std imports.

---

Outside diff comments:
In `@src/state_manager/cache.rs`:
- Around line 62-74: The hit/miss flag is inverted because the async closure
given to get_or_insert_async runs only on cache misses; change the logic so hit
starts true and the closure sets hit = false (so a computed entry marks a miss),
then keep the TIPSET_HIT / TIPSET_MISS increments as-is; update the mutable
variable usage around get_or_insert_async and the async closure that calls
compute() to flip the boolean assignment (use hit = true before calling, and
inside the async closure set hit = false).
🪄 Autofix (Beta)

Fix all unresolved CodeRabbit comments on this PR:

  • Push a commit to this branch (recommended)
  • Create a new PR with the fixes

ℹ️ Review info
⚙️ Run configuration

Configuration used: Repository UI

Review profile: CHILL

Plan: Pro

Run ID: f212dfa7-c1dc-4385-9772-3bcead221089

📥 Commits

Reviewing files that changed from the base of the PR and between 64e4d70 and cf4ebe4.

📒 Files selected for processing (2)
  • src/state_manager/cache.rs
  • src/utils/cache/size_tracking.rs

Comment thread src/utils/cache/size_tracking.rs Outdated
@hanabi1224
Copy link
Copy Markdown
Contributor Author

here are some documents where we are still haven't updated the lru cache metrics such as metrics.md. Please update those as well.

@akaladarshi docs updated

akaladarshi
akaladarshi previously approved these changes May 20, 2026
Copy link
Copy Markdown
Collaborator

@akaladarshi akaladarshi left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM. Nice Work!

just one comment regarding peek(BadBlockCache).

Comment thread src/chain/store/chain_store.rs Outdated
Comment thread src/shim/state_tree.rs
Comment thread docs/docs/users/guides/monitoring/metrics.md
Comment thread src/utils/cache/size_tracking.rs Outdated
Comment thread src/chain/store/chain_store.rs
Comment thread src/daemon/mod.rs Outdated
@hanabi1224 hanabi1224 enabled auto-merge May 21, 2026 13:50
@hanabi1224 hanabi1224 added this pull request to the merge queue May 21, 2026
Merged via the queue into main with commit 62c0289 May 21, 2026
43 checks passed
@hanabi1224 hanabi1224 deleted the hm/prefill-cache-collect-events branch May 21, 2026 14:19
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

RPC requires calibnet RPC checks to run on CI

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants