CI: ignore the "flaky" and "timing-dependent" test results

CI builds will now run these tests, but will ignore the results if they
fail. The relevant tests are ones that are sensitive to timing or
have edge conditions that make them more likely to fail on CI servers,
which are often heavily overloaded and slow.

This change only adds two additional tests to be ignored, since the
others already had the flaky keyword.

Closes #11865
This commit is contained in:
Dan Fandrich 2023-09-15 13:42:04 -07:00
parent 989e1f35e8
commit 589dca761c

View File

@ -71,8 +71,9 @@ TEST_F = -a -p -r
TEST_T = -a -t
TEST_E = -a -e
# !flaky means that it'll skip all tests using the flaky keyword
TEST_NF = -a -p !flaky
# ~<keyword> means that it will run all tests matching the keyword, but will
# ignore their results (since these ones are likely to fail for no good reason)
TEST_NF = -a -p ~flaky ~timing-dependent
# special CI target derived from nonflaky with CI-specific flags
TEST_CI = $(TEST_NF) -rm