Merged
Conversation
Noticed a few timeouts running eunit -j8 tests
big-r81
reviewed
Dec 22, 2025
big-r81
approved these changes
Dec 22, 2025
efd8910 to
3bb5cae
Compare
The main improvement is to separate the per-app makefile targets early, during templating phase, so they never share a common app structure during runtime. That lets us avoid reconfiguring the data directory at runtime, which effectively is an almost full restart internally since we're tearing down all the couch servers. So we're saving an extra app shutdown + restart cycle per app with the new scheme and improving the isolation. The new scheme uses the `tmp/$app` namespace for each app. After the tests run any unsuccessful `tmp/$app` directories will be left behind so they can be inspected. This is especially useful for looking at the failed couch.log entries. There is a separate catlogs target to concatenate failed app couch.log files though maybe keeping them separately may be even easier to inspect. Since if app a failed, we'd just rather look at app a's logs and not app c's logs. Since setup_eunit rebar template now takes an app argument, opted to remove some older eunit targets keeping only the new ones supporting parallel execution. Otherwise, would have to add a special case statement in the plugin for no-app vs one app cases. Previously, `make eunit apps=a,b,c` did not run apps `a`, `b` and `c` as separate goals in parallel execution. They were passed in as the `a,b,c` goal. To fix that parse the apps into individual space separated goals, so they can be sped up by parallel execution as well. To keep the apps as isolated from each other as possible make build_eunit_config also create separate etc folders, use separate BUILDDIR macros and each apps only uses its own fixtures. Also use 0-valued ports the eunit template to avoid any chance of port collision and resulting flakiness. Rename SUBDIRS to EUNIT_SUBDIRS since they used for eunit tests currently only. Also, exclude docs and fauxton apps from the target list, since they are not proper erlang apps and don't have eunit tests. Newer Erlangs have `file:del_dir_r/1` so use that in cleanup_dirs to save a few lines of code.
3bb5cae to
11bce02
Compare
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
A few improvements to the test suite to help it run better in parallel.
The main improvement is to separate the per-app makefile targets early, during templating phase, so they never share a common app structure during runtime. That lets us avoid reconfiguring the data directory at runtime, which effectively is an almost full restart internally since we're tearing down all the couch servers. So we're saving an extra app shutdown + restart cycle per app with the new scheme and improving the isolation.
The new scheme uses the
tmp/$appnamespace for each app. After the tests run any unsuccessfultmp/$appdirectories will be left behind so they can be inspected. This is especially useful for looking at the failed couch.log entries. There is a separate catlogs target to concatenate failed app couch.log files though maybe keeping them separately may be even easier to inspect. Since if app a failed, we'd just rather look at app a's logs and not app c's logs.Since setup_eunit rebar template now takes an app argument, opted to remove some older eunit targets keeping only the new ones supporting parallel execution. Otherwise, would have to add a special case statement in the plugin for no-app vs one app cases.
Previously,
make eunit apps=a,b,cdid not run appsa,bandcas separate goals in parallel execution. They were passed in as thea,b,cgoal. To fix that parse the apps into individual space separated goals, so they canbe sped up by parallel execution as well.
To keep the apps as isolated from each other as possible make build_eunit_config also create separate etc folders, use separate BUILDDIR macros and each apps only uses its own fixtures. Also use 0-valued ports theeunit template to avoid any chance of port collision and resulting flakiness.
Rename SUBDIRS to EUNIT_SUBDIRS since they used for eunit tests currently only. Also, exclude docs and fauxton apps from the target list, since they are not proper erlang apps and don't have eunit tests.
Newer Erlangs have
file:del_dir_r/1so use that in cleanup_dirs to save a few lines of code.There are a few other separate commits to update application with a few timeouts I noticed when running with higher concurrency. Another is make the replicator use its own fixture for attachments and not dip into couch app's fixtures folder since those could be setup up or torn down separately and we would rather not add intra-target dependencies during parallel runs.
Let's try running with
failfast=falsefor a bit in Jenkins since it might make it easier to debug failures. Once things settle, or if we see a lot of backlog from failing (doomed run) we can switch back.