tests: add the timing-dependent keyword on several tests

These are ones likely to fail on heavily-loaded machines that alter the
normal test timing. Most of these tests already had the flaky keyword
since this condition makes them more likely to fail on CI.
This commit is contained in:
Dan Fandrich 2023-09-07 23:48:33 -07:00
parent ae84a52c42
commit 381792dfbf
8 changed files with 20 additions and 3 deletions

View File

@ -190,7 +190,9 @@ tests. Try to use already used keywords. These keywords will be used for
statistical/informational purposes and for choosing or skipping classes of
tests. Keywords must begin with an alphabetic character, `-`, `[` or `{` and
may actually consist of multiple words separated by spaces which are treated
together as a single identifier.
together as a single identifier. Most keywords are only there to provide a way
for users to skip certain classes of tests, if desired, but a few are treated
specially by the test harness or build system.
When using curl built with Hyper, the keywords must include `HTTP` or `HTTPS`
for 'hyper mode' to kick in and make line ending checks work for tests.
@ -199,6 +201,12 @@ When running a unit test and the keywords include `unittest`, the `<tool>`
section can be left empty to use the standard unit test tool name `unitN` where
`N` is the test number.
The `text-ci` make target automatically skips test with the `flaky` keyword.
Tests that have strict timing dependencies have the `timing-dependent` keyword.
These are intended to eventually be treated specially on CI builds which are
often run on overloaded machines with unpredictable timing.
## `<reply>`
### `<data [nocheck="yes"] [sendzero="yes"] [base64="yes"] [hex="yes"] [nonewline="yes"] [crlf="yes"]>`

View File

@ -1,4 +1,5 @@
<testcase>
# This is the ftps equivalent of test1086
<info>
<keywords>
FTPS
@ -7,6 +8,7 @@ RETR
timeout
FAILURE
SLOWDOWN
timing-dependent
</keywords>
</info>

View File

@ -46,6 +46,7 @@ SIZE %TESTNUMBER
RETR %TESTNUMBER
QUIT
</protocol>
10 is CURLE_FTP_ACCEPT_FAILED
<errorcode>
10
</errorcode>

View File

@ -5,9 +5,10 @@ FTP
PORT
RETR
NODATACONN150
flaky
timeout
FAILURE
flaky
timing-dependent
</keywords>
</info>
# Server-side
@ -53,6 +54,7 @@ TYPE I
SIZE %TESTNUMBER
RETR %TESTNUMBER
</protocol>
# 28 is CURLE_OPERATION_TIMEDOUT
<errorcode>
28
</errorcode>

View File

@ -32,6 +32,7 @@ HTTP
HTTP PUT
Expect
flaky
timing-dependent
</keywords>
</info>
# Server-side

View File

@ -6,6 +6,7 @@ multi
resolve
speedcheck
flaky
timing-dependent
</keywords>
</info>

View File

@ -5,8 +5,9 @@ HTTP
HTTP GET
HTTP Basic auth
HTTP NTLM auth
flaky
NTLM
flaky
timing-dependent
</keywords>
</info>
# Server-side

View File

@ -3,6 +3,7 @@
<keywords>
unittest
cfilter
timing-dependent
</keywords>
</info>