Commit Graph

5442 Commits

Author SHA1 Message Date
Dan Fandrich
390af1ed5e runtests: refactor test runner code into runner.pm
This is code that is directly responsible for running a single test.
This will eventually run in a separate process as part of the parallel
testing project.

Ref: #10818
2023-04-18 13:18:17 -07:00
Dan Fandrich
bfa554b207 runtests: skip unneeded work if test won't be running
This speeds up tests by avoiding unnecessary processing.

Ref: #10818
2023-04-18 13:18:17 -07:00
Dan Fandrich
a3bccb2893 runtests: factor out singletest_postcheck
This will eventually need to be part of the test runner.

Ref: #10818
2023-04-18 13:18:17 -07:00
Dan Fandrich
cb5127e17c test303: kill server after test
Otherwise, an HTTP test closely following this one with a tight time
constraint (e.g. 672) could fail because the test server stays sitting
with the wait command for a while.
2023-04-18 13:18:17 -07:00
Sevan Janiyan
25dcbbcf7d tests/sshserver.pl: Define AddressFamily earlier
As the comment states "Address family must be specified before ListenAddress", otherwise the tests fail to run
`"failed starting SSH server" 52 times (582, 583, 600, 601, 602, 603, 604, 605, 606 and 43 more)`

Closes #10983
2023-04-18 12:50:12 -07:00
Daniel Stenberg
1c5ed24ee0
tests/keywords.pl: remove
This script does not work since the introduction of the test
preprocessing. If we need this functionality, it probably needs to be
moved into the runtests tool or similar.

Reported-by: Dan Fandrich
Fixes #10895
Closes #10987
2023-04-17 19:59:58 +02:00
Stefan Eissing
fc2f1e547a
http2: support HTTP/2 to forward proxies, non-tunneling
- with `--proxy-http2` allow h2 ALPN negotiation to
  forward proxies
- applies to http: requests against a https: proxy only,
  as https: requests will auto-tunnel
- adding a HTTP/1 request parser in http1.c
- removed h2h3.c
- using new request parser in nghttp2 and all h3 backends
- adding test 2603 for request parser
- adding h2 proxy test cases to test_10_*

scorecard.py: request scoring accidentally always run curl
with '-v'. Removed that, expect double numbers.

labeller: added http1.* and h2-proxy sources to detection

Closes #10967
2023-04-17 17:27:49 +02:00
Daniel Stenberg
17c71df421
tests/libtest/lib1900.c: remove
This file was left behind when the rest of the test was previously removed.

Follow-up to e50a877df7
2023-04-17 12:45:14 +02:00
Daniel Stenberg
54ac447b11
curl: do NOT append file name to path for upload when there's a query
Added test 425 to verify.

Reported-by: Dirk Rosenkranz
Bug: https://curl.se/mail/archive-2023-04/0008.html
Closes #10969
2023-04-15 10:52:33 +02:00
Stefan Eissing
5126cbda00
tests: make test_12_01 a bit more forgiving on connection counts 2023-04-13 08:46:53 +02:00
Daniel Stenberg
4cfa5bcc9a
urlapi: cleanups
- move host checks together
- simplify the scheme parser loop and the end of host name parser
- avoid itermediate buffer storing in multiple places
- reduce scope for several variables
- skip the Curl_dyn_tail() call for speed
- detect IPv6 earlier and skip extra checks for such hosts
- normalize directly in dynbuf instead of itermediate buffer
- split out the IPv6 parser into its own funciton
- call the IPv6 parser directly for ipv6 addresses
- remove (unused) special treatment of % in host names
- junkscan() once in the beginning instead of scattered
- make junkscan return error code
- remove unused query management from dedotdotify()
- make Curl_parse_login_details use memchr
- more use of memchr() instead of strchr() and less strlen() calls
- make junkscan check and return the URL length

An optimized build runs one of my benchmark URL parsing programs ~41%
faster using this branch. (compared against the shipped 7.88.1 library
in Debian)

Closes #10935
2023-04-13 08:41:40 +02:00
Dan Fandrich
8e75c4f978 pathhelp: use the cached $use_cygpath when available 2023-04-11 14:55:32 -07:00
Dan Fandrich
def8dc8071 runtests: eliminate unneeded variable 2023-04-11 14:55:32 -07:00
Dan Fandrich
a3605d6a56 runtests: make the # of server start attempts a constant 2023-04-11 14:55:32 -07:00
Dan Fandrich
44256902f3 runtests: on startup failure call displaylogs only in serverfortest
This reduces the number of calls spread throughout the code.

Ref: #10818
Closes #10919
2023-04-11 14:55:32 -07:00
Dan Fandrich
f82a38e0f3 runtests: return an error code with startservers()
The code indicates the kind of failure encountered in starting a server,
which can be used by the caller to tailor the user experience.

Ref: #10818
2023-04-11 14:55:32 -07:00
Dan Fandrich
19b062a492 runtests: abort early if runpingpongserver is given a bad server type 2023-04-11 14:55:32 -07:00
Dan Fandrich
f25627014e runtests: don't use the SMB server verification time as reference
%FTPTIME2 and %FTPTIME3 should be set by the FTP server only, for
consistency.
2023-04-11 14:55:32 -07:00
Dan Fandrich
d39db81192 tests: factor out the test server management code
This now lives in servers.pm with some configuration variables moved to
globalconfig.pm

Ref: #10818
2023-04-11 14:55:32 -07:00
Dan Fandrich
a2ce734e28 runtests: remove an inappropriate use of runclientoutput
This function is intended for running client code, not servers.
2023-04-11 14:55:32 -07:00
Dan Fandrich
fdb24b3916 runtests: only add $LIBDIR to the path for checktestcmd
Since checkcmd is for finding servers, there will never be anything in
this directory of interest to them.

Ref: #10818
2023-04-11 14:55:32 -07:00
Dan Fandrich
dee50c9c51 tests: log sshserver.pl messages to a file
The logmsg messages were thrown away before, so they are now available
for debugging.
2023-04-11 14:55:32 -07:00
Dan Fandrich
25aba1683a runtests: also show DISABLED tests with -l
Other reasons for skipping tests are ignored for -l, so being explicitly
disabled should be too.
2023-04-11 14:55:32 -07:00
Dan Fandrich
89ede3953b runtests: move the UNIX sockets into $PIDDIR
These were missed when the other server files were moved there.

Follow-up to 70d2fca2

Ref: #10818
2023-04-11 14:55:31 -07:00
Dan Fandrich
44677b6565 tests: tighten up perl exports
This reduces namespace pollution a little.

Ref: #10818
2023-04-11 14:55:31 -07:00
Dan Fandrich
efbaa612f7 tests: turn perl modules into full packages
This helps enforce more modularization and encapsulation. Enable and fix
warnings on a few packages.  Also, rename ftp.pm to processhelp.pm since
there's really nothing ftp-specific in it.

Ref: #10818
2023-04-11 14:55:31 -07:00
Dan Fandrich
dc18b40b40 tests: log a too-long Unix socket path in sws and socksd
Ref: #10919
2023-04-11 10:10:16 -07:00
Daniel Stenberg
309a517ffd
lib1560: verify that more bad host names are rejected
when setting the hostname component of a URL

Closes #10922
2023-04-11 11:33:07 +02:00
Dan Fandrich
52d4a41b2f test1169: fix so it works properly everywhere
- Use an absolute path for the -L option since the module isn't in the
  perl path
- Create the needed test file in a <file> section; <precheck> isn't
  intended for this
- Fix the test number in the file name, which was wrong

Follow-up to f754990a

Ref: #10818
Fixes #10889
Closes #10917
2023-04-10 11:50:51 -07:00
Dan Fandrich
6ed0629901 tests: stop using strndup(), which isn't portable
It's not available on Solaris 10, for example. Since this is just test
code that doesn't need to use an optimized system version, replace it
with the implementation copied from tool_cb_hdr.c.
2023-04-10 10:51:07 -07:00
Dan Fandrich
a72a548774 runtests: fix an incorrect comment about the ld_preload feature
Follow-up to 1f631864

Ref: #10818
2023-04-10 10:51:07 -07:00
Daniel Stenberg
826e8011d5
urlapi: prevent setting invalid schemes with *url_set()
A typical mistake would be to try to set "https://" - including the
separator - this is now rejected as that would then lead to
url_get(... URL...) would get an invalid URL extracted.

Extended test 1560 to verify.

Closes #10911
2023-04-09 23:23:54 +02:00
Daniel Stenberg
5f307986a3
lib: remove CURLX_NO_MEMORY_CALLBACKS
The only user of this define was 'chkdecimalpoint' - a special purpose
test tool that was built but not used anymore (since 17c18fbc3 - Apr
2020).

Closes #10908
2023-04-08 17:47:10 +02:00
Daniel Stenberg
4033642930
content_encoding: only do tranfer-encoding compression if asked to
To reduce surprises. Update test 387 and 418 accordingly.

Closes #10899
2023-04-07 13:39:20 +02:00
Daniel Stenberg
3de6b6cfe5
sws: comparison of unsigned expression < 0 is always false
Follow-up to 356dd0b73a

Closes #10903
2023-04-07 12:40:07 +02:00
Stefan Eissing
356dd0b73a
tests: increase sws timeout for more robust testing
- for https CONNECT forwarding, this was fixed at 5 seconds
  which led to spurious CI test failures
- add --keepalive parameter to sws to control this
- let httpserver use 30 seconds

Closes #10898
2023-04-06 16:28:34 +02:00
Stefan Eissing
4ae2d9f24d
proxy: http2 proxy tunnel implementation
- currently only on debug build and when env variable
  CURL_PROXY_TUNNEL_H2 is present.
- will ALPN negotiate with the proxy server and switch
  tunnel filter based on the protocol negotiated.
- http/1.1 tunnel code moved into cf-h1-proxy.[ch]
- http/2 tunnel code implemented in cf-h2-proxy.[ch]
- tunnel start and ALPN set remains in http_proxy.c
- moving all haproxy related code into cf-haproxy.[ch]

VTLS changes
- SSL filters rely solely on the "alpn" specification they
  are created with and no longer check conn->bits.tls_enable_alpn.
- checks on which ALPN specification to use (or none at all) are
  done in vtls.c when creating the filter.

Testing
- added a nghttpx forward proxy to the pytest setup that
  speaks HTTP/2 and forwards all requests to the Apache httpd
  forward proxy server.
- extending test coverage in test_10 cases
- adding proxy tests for direct/tunnel h1/h2 use of basic auth.
- adding test for http/1.1 and h2 proxy tunneling to pytest

Closes #10780
2023-04-06 13:04:46 +02:00
Stefan Eissing
8f50e393ab
vtls and h2 improvements
- eliminate receive loop in vtls to fill buffer. This may
  lead to partial reads of data which is counter productive
- let http2 instead loop smarter to process pending network
  data without transfer switches

scorecard improvements
- do not start caddy when only httpd is requested
- allow curl -v to stderr file on --curl-verbose

Closes #10891
2023-04-06 09:41:54 +02:00
Daniel Stenberg
3da642c4f0
tests: 1078 1288 1297 use valid IPv4 addresses
With the enhanced URL parser, these tests failed because of their bad
IPv4 use.
2023-04-06 09:02:10 +02:00
Daniel Stenberg
17a15d8846
urlapi: detect and error on illegal IPv4 addresses
Using bad numbers in an IPv4 numerical address now returns
CURLUE_BAD_HOSTNAME.

I noticed while working on trurl and it was originally reported here:
https://github.com/curl/trurl/issues/78

Updated test 1560 accordingly.

Closes #10894
2023-04-06 09:02:00 +02:00
Daniel Stenberg
f042e1e75d
urlapi: URL encoding for the URL missed the fragment
Meaning that it would wrongly still store the fragment using spaces
instead of %20 if allowing space while also asking for URL encoding.

Discovered when playing with trurl.

Added test to lib1560 to verify the fix.

Closes #10887
2023-04-05 08:30:12 +02:00
Daniel Stenberg
808cb31756
tool_writeout: add URL component variables
Output specific components from the used URL. The following variables
are added for this purpose:

  url.scheme, url.user, url.password, url.options, url.host, url.port,
  url.path, url.query, url.fragment, url.zoneid

Add the following for outputting parts of the "effective URL":

 urle.scheme, urle.user, urle.password, urle.options, urle.host, urle.port,
 urle.path, urle.query, urle.fragment, urle.zoneid

Added test 423 and 424 to verify.

Closes #10853
2023-04-04 14:42:44 +02:00
Stefan Eissing
e0c3424fb1
tests/http: improved httpd detection
- better error messages when not found/complete
- handling of `--without-test-httpd`

Reported-by: kwind on github
Fixes #10879
Closes #10883
2023-04-04 14:37:08 +02:00
Dan Fandrich
a13ef31d0f unit tests: use the unit test infrastructure better
Allow UNITTEST_STOP to return the error code, use the fail & abort
macros to indicate test failure and return success instead of fail if
the unit test can't test anything because of missing features at
compile-time.  A couple of tests could never fail because they were
overriding the failure return code.
2023-04-01 08:59:38 -07:00
Dan Fandrich
419a745da6 runtests: strip EOL on precheck output on Windows, too
Precheck failures would show on two lines in the test summary output
otherwise.
2023-03-31 23:08:00 -07:00
Dan Fandrich
70d2fca2f6 tests: move server config files under the pid dir
These files are generated by the test servers and must therefore be
found in the log directory to make them available to only those servers
once multiple test runners are executing in parallel. They must also not
be deleted with the log files, so they are stored in the pidfile
directory.

Ref: #10818
Closes #10875
2023-03-31 23:08:00 -07:00
Dan Fandrich
1cffced9c5 runtests: use the ssh key filenames from the sshhelp package 2023-03-31 23:08:00 -07:00
Dan Fandrich
3cc41a2d36 tests: move pidfiles and portfiles under the log directory
This is to segregate all files written by a test process into a single
root to allow for future parallel testing.

Ref: #10818
Closes #10874
2023-03-31 23:08:00 -07:00
Dan Fandrich
ec2a5bc5fc runtests: minor code cleanups 2023-03-31 23:08:00 -07:00
Dan Fandrich
62ba1d6bd9 runtests: call processexists() and pidfromfile()
rather than duplicating the logic in several places.
2023-03-31 23:08:00 -07:00