Reduce VCR recording runtime after package split#9722
Merged
shuyama1 merged 1 commit intoDec 28, 2023
Conversation
Collaborator
|
Hi there, I'm the Modular magician. I've detected the following information about your changes: Diff reportYour PR hasn't generated any diffs, but I'll let you know if a future commit does. |
slevenick
approved these changes
Dec 28, 2023
This was referenced Dec 28, 2023
kapreus
pushed a commit
to kapreus/magic-modules
that referenced
this pull request
Jan 2, 2024
kylase
pushed a commit
to yuanchuankee/magic-modules
that referenced
this pull request
Jan 21, 2024
balanaguharsha
pushed a commit
to balanaguharsha/magic-modules
that referenced
this pull request
May 2, 2024
pengq-google
pushed a commit
to pengq-google/magic-modules
that referenced
this pull request
May 21, 2024
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
3 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
I've observed some significant recording runtime increases, especially for large size of affected tests, after VCR split.
Example PR: #9666
Before VCR split: ~20 mins for recording #9666 (comment)
After VCR split: ~2 hours for recording #9666 (comment)
What I suspect is, even though VCR split only changed how we run tests in the REPLAYING mode, it skips package compiling for all non-affected services and therefore it needs to compile them (probably multiple times) in the RECORDING mode (previously using compilation cache from the REPLAYING)
Given how we enable the parallelism in RECORDING and the lower size of it due to the limitation of Terraform log setup, it needs much more total compilation time and therefore leads to the long RECORDING runtime.
To resolve this issue, since we've known what packages are affected and needed to run tests against, we can only loop through these packages and run failed tests against them.
Testing: #9717 (comment): recording mode took ~2.5h for 12
TestAccComputeFirewall*and after apply the change in this PR, it takes only ~10 minutes #9721 (comment).Release Note Template for Downstream PRs (will be copied)