Commit Graph

30290 Commits

Author SHA1 Message Date
Dan Fandrich
a4aebd73e2 smbserver: remove temporary files before exit
Each execution of test 1451 would leave a file in /tmp before. Since
Windows can't delete a file while it's open, all the temporary file
names are stored and deleted on exit.

Closes #10990
2023-04-25 16:12:20 -07:00
Stefan Eissing
930c00c259
Websocket en-/decoding
- state is fully kept at connection, since curl_ws_send() and
  curl_ws_rec() have lifetime beyond usual transfers
- no more limit on frame sizes

Reported-by: simplerobot on github
Fixes #10962
Closes #10999
2023-04-25 23:16:51 +02:00
Patrick Monnerat
3f0b81c112 urldata: copy CURLOPT_AWS_SIGV4 value on handle duplication
Prior to this change STRING_AWS_SIGV4 (CURLOPT_AWS_SIGV4) was wrongly
marked as binary data that could not be duplicated.

Without this fix, this option's value is not copied upon calling
curl_easy_duphandle().

Closes https://github.com/curl/curl/pull/11021
2023-04-25 14:59:12 -04:00
Stefan Eissing
3b7a8a25b6
http3: expire unpaused transfers in all HTTP/3 backends
Closes #11005
2023-04-25 17:50:57 +02:00
Stefan Eissing
5622e431ed
http2: always EXPIRE_RUN_NOW unpaused http/2 transfers
- just increasing the http/2 flow window does not necessarily
  make a server send new data. It may already have exhausted
  the window before

Closes #11005
2023-04-25 17:50:53 +02:00
Stefan Eissing
fbea71f293
http2: pass stream to http2_handle_stream_close to avoid NULL checks
Closes #11005
2023-04-25 17:50:45 +02:00
Stefan Eissing
cab2d56ea5
h2/h3: replace state.drain counter with state.dselect_bits
- `drain` was used by http/2 and http/3 implementations to indicate
  that the transfer requires send/recv independant from its socket
  poll state. Intended as a counter, it was used as bool flag only.
- a similar mechanism exists on `connectdata->cselect_bits` where
  specific protocols can indicate something similar, only for the
  whole connection.
- `cselect_bits` are cleard in transfer.c on use and, importantly,
  also set when the transfer loop expended its `maxloops` tries.
  `drain` was not cleared by transfer and the http2/3 implementations
  had to take care of that.
- `dselect_bits` is cleared *and* set by the transfer loop. http2/3
  does no longer clear it, only set when new events happen.

This change unifies the handling of socket poll overrides, extending
`cselect_bits` by a easy handle specific value and a common treatment in
transfers.

Closes #11005
2023-04-25 17:49:28 +02:00
Daniel Stenberg
a97e4eb95f
socketpair: verify with a random value
... instead of using the curl time struct, since it would use a few
uninitialized bytes and the sanitizers would complain. This is a neater
approach I think.

Reported-by: Boris Kuschel
Fixes #10993
Closes #11015
2023-04-25 17:40:15 +02:00
Stefan Eissing
2079cb26a1
HTTP3: document the ngtcp2/nghttp3 versions to use for building curl
- refs #11011 to clarify this for people building curl themselves

Closes #11019
2023-04-25 17:38:59 +02:00
Daniel Stenberg
7815647d65
lib: unify the upload/method handling
By making sure we set state.upload based on the set.method value and not
independently as set.upload, we reduce confusion and mixup risks, both
internally and externally.

Closes #11017
2023-04-25 12:38:38 +02:00
Daniel Stenberg
e5af5b7905
RELEASE-NOTES: synced 2023-04-25 09:49:41 +02:00
Dan Fandrich
fe61f015e5 CI: don't run CI jobs if only another CI was changed
A few paths were missed in the last commit, as well as a job added since
then.

Followup-to 395b9175
2023-04-24 20:46:09 -07:00
Dan Fandrich
6d0d8b90e2 CI: adjust labeler match patterns 2023-04-24 20:46:09 -07:00
Dan Fandrich
189f9e23b6 runtests: support buffering log messages in runner & servers
Log messages generated with logmsg can now be buffered and returned from
the runner as a return value.  This will be needed with parallel testing
to allow all messages for one test to be displayed together instead of
interspersed with messages of multiple tests. Buffering can be disabled
by setting a logging callback function with setlogfunc, which is
currently being done to preserve existing logging behaviour for now.

Some additional output is generated in verbose and debugprotocol modes,
which don't always use logmsg. These modes also impact some servers
which generate extra messages. No attempt is made to buffer everything
if these modes are enabled.

Ref: #10818
Closes #11016
2023-04-24 16:28:23 -07:00
Dan Fandrich
c3453dcb9d runtests: more consistently use logmsg in server control code
Also, display an error when sshversioninfo returns one.

Ref: #10818
2023-04-24 16:04:24 -07:00
Dan Fandrich
b88ea272c1 runtests: create runner functions for clearlocks and stopservers
runtests.pl now uses runner for all server actions beyond the initial
variable configuration.

Ref: #10818
2023-04-24 16:04:24 -07:00
Dan Fandrich
640f4c2267 runtests: tightened servers package exports
The defaults are intended for runtests.pl, whereas runner.pm needs to
explicitly specify them.
2023-04-24 16:04:24 -07:00
Dan Fandrich
8da49c9e38 runtests: display logs on server failure in singletest()
This is closer to the place where logs are displayed on test failure.
Also, only display these logs if -p is given, which is the same flag
that controls display of test failure logs. Some server log files
need to be deleted later so that they stay around long enough to be
displayed on failure.

Ref: #10818
2023-04-24 16:03:44 -07:00
Dan Fandrich
90158f0bab runtests: turn a print into a logmsg
Also enable another couple of useful messages in verbose mode.

Ref: #10818
2023-04-24 12:13:17 -07:00
Daniel Stenberg
406cc24f5d
http: store the password in the correct variable
Typo from fc2f1e547a, detected by Coverity (because there's dead code
due to this).

Closes #11002
2023-04-24 13:54:52 +02:00
Stefan Eissing
20252b77bc
HTTP3/quiche: terminate h1 response header when no body is sent
- fixes a failure in test2501 where a response without body was missing
  the final empty line

Closes #11003
2023-04-24 13:40:47 +02:00
Dan Fandrich
a8706fd8d0 runtests: move showdiff into runtests.pl
It's not used anywhere else.
2023-04-22 13:07:35 -07:00
Dan Fandrich
20fa5b74a5 devtest: add a new script for testing the test harness
This is currently useful for starting a test server on its own without
an associated test, which can be used for interactive curl testing or
for validating parts of the test harness itself. More commands can be
added to perform additional functions in the future.

Ref: #10818
Closes #11008
2023-04-22 13:07:35 -07:00
Dan Fandrich
a549e046b1 runtests: refactor the main test loop into two
The test loop now has an initial loop that first runs through all
possible tests to build a set of those to attempt on this run based on
features and keywords and only then goes through that new list to run
them.  This actually makes it three loops through all tests cases, as
there is an existing loop that gathers possible test numbers from the
test files on disk.

This has two minor effects on the output: all the tests that will be
skipped are displayed at the start (instead of being interspersed with
other tests) and the -l option no longer shows a count of tests at the
end or a (misleading) statement that tests have run successfully. The
skipped tests are also omitted from the test results sent to AppVeyor
and Azure in CI builds.

Another effect is a reduction in the amount of work considered part of
the "Test definition reading and preparation time" reported with -r
making those figures slightly lower than before.

Ref: #10818
2023-04-22 13:07:35 -07:00
Dan Fandrich
020cf1c117 runtests: track only the current test timings in runner.pm
This avoids passing these data through through global variables, which
soon won't be possible.

Ref: #10818
2023-04-22 12:58:04 -07:00
Dan Fandrich
c6e7f6c61f runtests: skip test preprocessing when doing -l
This speeds up the output tremendously by avoiding unnecessary work.
2023-04-22 12:58:03 -07:00
Dan Fandrich
6210bc0111 runtests: simplify value returned regarding use of valgrind
As a side effect this will now also show in verbose mode that valgrind
is being skipped on tests that explicitly disable it, such as 600.

Ref: #10818
2023-04-22 12:58:03 -07:00
Dan Fandrich
4a41745e21 runtests: fix quoting in Appveyor and Azure test integration
Test 1442's name was not quoted correctly so wasn't registered in
Appveyor and it had the wrong name in Azure. The JSON string quotes were
also invalid, even though both servers happened to accept it regardless.

Closes #11010
2023-04-22 11:50:03 -07:00
Daniel Stenberg
6b1e4dc6cd
RELEASE-NOTES: synced 2023-04-19 08:12:57 +02:00
Dan Fandrich
47f2e556aa runtests: spread out the port numbers used by servers
The server ports are chosen randomly for each server, but the random
ranges chosen were inconsistently-sized and overlapping. Now, they are
spread out more so at least the first random port chosen for each server
is guaranteed to not also be chosen by another server. The starting port
numbers are also raised to put them in the Ephemeral Port range—not the
range defined by RFC 6335 but the one used by Linux, which starts lower
and gives us more room to work with.

Reported-by: Daniel Stenberg
2023-04-18 13:18:17 -07:00
Dan Fandrich
04113319db runtests: fix problems on <killserver> failure
The verify time must be set in this case, like all cases. An error
message needs to be displayed as well.
2023-04-18 13:18:17 -07:00
Dan Fandrich
1f6a9f48aa runtests: fix perl warning when <tool> is wrong 2023-04-18 13:18:17 -07:00
Dan Fandrich
f9e8c5f578 runtests: don't try to stop stunnel before trying again
Calling stopserver() before retrying stunnel due to an error would stop
the dependent server (such as HTTP) meaning stunnel would have nothing
to talk to when it came up. Don't try to force a stop when it didn't
actually start.  Also, don't mark the server as bad for future use when
it starts up on a retry.

Reported-by: eaglegai at github
Tested-by: eaglegai at github
Fixes #10976
2023-04-18 13:18:17 -07:00
Dan Fandrich
2e0b70b8bc runtests: don't accidentally randomly choose the same port
If a server couldn't be started on a port, a new one is randomly chosen
and the server is tried again. Avoid accidentally using a
randomly-chosen 0 port offset by adding 1 to the random number.

Found-by: Daniel Stenberg
2023-04-18 13:18:17 -07:00
Dan Fandrich
b11840807b runtests: don't attempt to use a port we know is in use
This reduces the startup time when there is a known conflict on the
random port chosen for a server.  This was already done for stunnel, but
now it's done for all servers.
2023-04-18 13:18:17 -07:00
Dan Fandrich
200c409021 http-server: fix server name in a log message
This changed when the file was renamed in commit cbf57176
2023-04-18 13:18:17 -07:00
Dan Fandrich
707f74c04a runtests: refactor into more packages
testutil.pm now contains a few miscellaneous functions that are used in
several places but have no better place to live.  subvariables moves to
servers.pm since most variables that it substitutes relate to servers,
so this is the most appropriate place. Rename a few functions for better
naming consistency.

Ref: #10818
Closes #10995
2023-04-18 13:18:17 -07:00
Dan Fandrich
ba51b39732 runtests: call timestampskippedevents() in singletest
..rather than by the runner
2023-04-18 13:18:17 -07:00
Dan Fandrich
43b876d7df runtests: assume a newer Valgrind by default
The tests for an older Valgrind version should probably just be deleted,
given that they're testing for an 18-year-old version.
2023-04-18 13:18:17 -07:00
Dan Fandrich
390af1ed5e runtests: refactor test runner code into runner.pm
This is code that is directly responsible for running a single test.
This will eventually run in a separate process as part of the parallel
testing project.

Ref: #10818
2023-04-18 13:18:17 -07:00
Dan Fandrich
bfa554b207 runtests: skip unneeded work if test won't be running
This speeds up tests by avoiding unnecessary processing.

Ref: #10818
2023-04-18 13:18:17 -07:00
Dan Fandrich
a3bccb2893 runtests: factor out singletest_postcheck
This will eventually need to be part of the test runner.

Ref: #10818
2023-04-18 13:18:17 -07:00
Dan Fandrich
cb5127e17c test303: kill server after test
Otherwise, an HTTP test closely following this one with a tight time
constraint (e.g. 672) could fail because the test server stays sitting
with the wait command for a while.
2023-04-18 13:18:17 -07:00
Patrick Monnerat
7c142d0571 OS400: provide ILE/RPG usage examples
Closes https://github.com/curl/curl/pull/10994
2023-04-18 16:14:00 -04:00
Patrick Monnerat
59ce2620a9 OS400: improve vararg emulation
- Use V7R4 RPG procedure overloading to improve vararg emulation.

From OS400 V7R4 and above, ILE/RPG implements a limited procedure
overloading feature that can be used to improve curl's typed
implementation of varargs procedures. This commit applies it to
curl_easy_setopt(), curl_multi_setopt(), curl_share_setopt() and
curl_easy_getinfo().

Closes https://github.com/curl/curl/pull/10994
2023-04-18 16:13:58 -04:00
Patrick Monnerat
faa04801db OS400: fix and complete ILE/RPG binding
- Fix wrong definitions of CURL_ZERO_TERNINATED, curl_mime_data() and
  curl_mime_data_ccsid().

- Add recent definitions, in particular blob, header API and WebSockets
  API.

- Support for CURLVERSION_ELEVENTH.

- New functions for EBCDIC support.

Reflect these changes in README.OS400.

Closes https://github.com/curl/curl/pull/10994
2023-04-18 16:13:57 -04:00
Patrick Monnerat
a1fa2b30c0 OS400: implement EBCDIC support for recent features
- Support CURLVERSION_ELEVENTH.

- New function curl_url_strerror_ccsid().

- curl_easy_setopt_ccsid() supports blobs and 3 recent string options.

- New function curl_easy_header_ccsid().

- New generic latin1<-->ccsid conversion functions curl_from_ccsid() and
  curl_to_ccsid() for user convenience.

- README.OS400 updated accordingly.

- Removed a leftover QsoSSL support identifier.

Closes https://github.com/curl/curl/pull/10994
2023-04-18 16:13:56 -04:00
Patrick Monnerat
b98db94267 OS400: rework build scripts
- Rename shell function "system" to "CLcommand" to avoid confusion with
  built-in command.

- Reformat scripts. Fix some indentations. Avoid lines > 80 characters
  where possible.

- Support ASCII runtime development files in a user-defined directory
  path.

- FIX SONAME detection.

- Drop form API test program compilation (does not exist anymore).

Closes https://github.com/curl/curl/pull/10994
2023-04-18 16:13:55 -04:00
Sevan Janiyan
25dcbbcf7d tests/sshserver.pl: Define AddressFamily earlier
As the comment states "Address family must be specified before ListenAddress", otherwise the tests fail to run
`"failed starting SSH server" 52 times (582, 583, 600, 601, 602, 603, 604, 605, 606 and 43 more)`

Closes #10983
2023-04-18 12:50:12 -07:00
Stefan Eissing
db82878afd quiche: Enable IDLE egress handling
Follow-up to 544abeea which added the handling but wrongly left it
commented out.

Closes https://github.com/curl/curl/pull/11000
2023-04-18 15:35:38 -04:00