Speed up tests - #23442
Draft
jelly wants to merge 5 commits into
Draft
Conversation
Originally added in 504a21a but no longer applicable in the current version of the page.
The initial boot is already a "reboot" from the initial provision, so this can be skipped as its quite costly. A reboot costs ~ 30 seconds on in CI.
In CI these tests take 3 and a half minutes in total, while a standard test run already tests `--script` and all tests run in the same container so the same qemu/python is used for testing. Fedora CoreOS was picked as it is the required passing scenario which also runs this test.
This test only executes TestTodo.testTodo which is a simple `unittest.TestCase` so it can be nondestructive. This skips bringing up a new virtual machine which is quite expensive (takes ~ 30 seconds in CI).
These tests test the TAP / run-tests API so don't need to be run on every image, especially as TestRunTest.testRetry takes 5 minutes to run.
Member
Author
|
The speedup is less then desired because of our mass parallelism (8 cores). |
Member
Author
|
@allisonkarlitskaya review would still be nice, what do you think? |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
This should improve the other scenario by only running our testapi/bots api tests on Fedora CoreOS only saving 8 minutes per "other" scenario.
Other commits are small test improvements to reduce doing unneeded work.
Baseline without retries:
arch/other - 22 minutes
centos-9-bootc/other ~ 20 minutes
debian-testing/other ~ 22 minutes
fedora-44/other ~ 24 minutes
fedora-core/other ~ 21 minutes