Split the subgraph_deployment table into two #6003
Merged
Conversation
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
8e47a78 to
899b13a
Compare
zorancv
approved these changes
May 16, 2025
| .load::<(Deployment, Head)>(conn)? | ||
| } | ||
| .into_iter() | ||
| .map(DeploymentDetail::from) |
This also makes the timeouts used for IPFS requests configurable; the default of 1s in debug builds is too short for the runner tests in CI and we therefore set it to the 60s for release builds for those tests.
We split the table `subgraphs.subgraph_deployment` into two tables, `subgraphs.head` and `subgraphs.deployment` where the `head` table only contains the metadata that changes on every block. This should help with situations where the `subgraph_deployment` table gets very bloated since the `head` table that gets bloated through frequent changes has much smaller rows than the current `subgraph_deployment` table. Rows in `subgraph_deployment` can grow as big as 500k, whereas rows in the `head` table will only take about 350 bytes at most. Updates will also be marginally better on the `heads` table since it only has one index rather than the two that `subgraph_deployment` has.
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
We split the table
subgraphs.subgraph_deploymentinto two tables,subgraphs.headandsubgraphs.deploymentwhere theheadtable only contains the metadata that changes on every block.This should help with situations where the
subgraph_deploymenttable gets very bloated since theheadtable that gets bloated through frequent changes has much smaller rows than the currentsubgraph_deploymenttable. Rows insubgraph_deploymentcan grow as big as 500k, whereas rows in theheadtable will only take about 350 bytes at most.Updates will also be marginally better on the
headstable since it only has one index rather than the two thatsubgraph_deploymenthas.This change has the downstream effect that there is no more
sharded.subgraph_deploymenttable. Instead there are nowsharded.headandsharded.deploymenttables. This change will also require that all dashboards or other tools that used to accesssubgraph_deploymentnow access the two new tables.Besides splitting tables, the new tables also use
int4columns throughout for block numbers rather thannumericsince the rest of the system is restricted toint4block numbers everywhere else.