Make all store/db interactions async #6194
Merged
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
We have a lot of places where we do blocking database work from a non-blocking tokio task. This PR addresses this longstanding issue by using async database connections and
diesel_async. With that, we don't need to rely on error-prone approaches that require spawning blocking tasks whenever we do database work.Most of this PR is concerned with sprinkling async/await throughout the code base. There are a lot of commits, but most of them are boring, and I tried to structure it so that many of them just add async/await in a few places to make them easy to review individually.
Since switching to async connections also requires changing the connection pool, towards the end of the PR there are changes to recreate the instrumentation we had with r2d2 with deadpool.
There are still places in the codebase where
spawn_blockingis used, but those can be removed in separate PR's as this one is already crazy long. My apologies to the reviewer(s)I made sure that
cargo checkpassed after each commit; in addition, I ran CI after every few (say 5-10) commits and made sure that passed, too, up to the commit markedSQUASH FROM HEREas it was not possible to pass CI from that point on untilCI PASSES AGAINas theAsyncConnectionWrapperusesblock_onfor db interactions, and too many of them happen on the main tokio threads. For those, tokio panics as you are not allowed to useblock_onon those threads.Fixes #905