Commit Graph

5061 Commits

Author SHA1 Message Date
Mats Lindestam
d1e7d9197b
libssh2: add SHA256 fingerprint support
Added support for SHA256 fingerprint in command line curl and in
libcurl.

Closes #7646
2021-09-26 23:20:53 +02:00
Rikard Falkeborn
e75be2c4b2
cleanup: constify unmodified static structs
Constify a number of static structs that are never modified. Make them
const to show this.

Closes #7759
2021-09-23 12:54:35 +02:00
Daniel Stenberg
883cf374d8
test897: verify delivery of IMAP post-body header content
The "content" is delivered as "body" by curl, but the envelope continues
after the body and the rest of it should be delivered as header.

The IMAP server can now get 'POSTFETCH' set to include more data to
include after the body and test 897 is done to verify that such "extra"
header data is in fact delivered by curl as header.

Ref: #7284 but fails to reproduce the issue

Closes #7748
2021-09-21 08:18:42 +02:00
Daniel Stenberg
beb8990d93
http: fix the broken >3 digit response code detection
When the "reason phrase" in the HTTP status line starts with a digit,
that was treated as the forth response code digit and curl would claim
the response to be non-compliant.

Added test 1466 to verify this case.

Regression brought by 5dc594e44f
Reported-by: Glenn de boer
Fixes #7738
Closes #7739
2021-09-17 17:52:00 +02:00
Daniel Stenberg
da2598f823
hsts: CURLSTS_FAIL from hsts read callback should fail transfer
... and have CURLE_ABORTED_BY_CALLBACK returned.

Extended test 1915 to verify.

Reported-by: Jonathan Cardoso
Fixes #7726
Closes #7729
2021-09-16 12:36:12 +02:00
Daniel Stenberg
5b7aaefc07
test1184: disable
The test should be fine and it works for me repeated when run manually,
but clearly it causes CI failures and it needs more research.

Reported-by: RiderALT on github
Fixes #7725
Closes #7732
2021-09-16 12:16:48 +02:00
Kamil Dudka
ab78d2c679 tests/sshserver.pl: make it work with openssh-8.7p1
... by not using options with no argument where an argument is required:

=== Start of file tests/log/ssh_server.log
curl_sshd_config line 6: no argument after keyword "DenyGroups"
curl_sshd_config line 7: no argument after keyword "AllowGroups"
curl_sshd_config line 10: Deprecated option AuthorizedKeysFile2
curl_sshd_config line 29: Deprecated option KeyRegenerationInterval
curl_sshd_config line 39: Deprecated option RhostsRSAAuthentication
curl_sshd_config line 40: Deprecated option RSAAuthentication
curl_sshd_config line 41: Deprecated option ServerKeyBits
curl_sshd_config line 45: Deprecated option UseLogin
curl_sshd_config line 56: no argument after keyword "AcceptEnv"
curl_sshd_config: terminating, 3 bad configuration options
=== End of file tests/log/ssh_server.log

=== Start of file log/sftp_server.log
curl_sftp_config line 33: Unsupported option "rhostsrsaauthentication"
curl_sftp_config line 34: Unsupported option "rsaauthentication"
curl_sftp_config line 52: no argument after keyword "sendenv"
curl_sftp_config: terminating, 1 bad configuration options
Connection closed.
Connection closed
=== End of file log/sftp_server.log

Closes #7724
2021-09-16 08:37:25 +02:00
Daniel Stenberg
54f6d836d8
hsts: handle unlimited expiry
When setting a blank expire string, meaning unlimited, curl would pass
TIME_T_MAX to getime_r() when creating the output, while on 64 bit
systems such a large value cannot be convetered to a tm struct making
curl to exit the loop with an error instead. It can't be converted
because the year it would represent doesn't fit in the 'int tm_year'
field!

Starting now, unlimited expiry is instead handled differently by using a
human readable expiry date spelled out as "unlimited" instead of trying
to use a distant actual date.

Test 1660 and 1915 have been updated to help verify this change.

Reported-by: Jonathan Cardoso
Fixes #7720
Closes #7721
2021-09-15 14:37:36 +02:00
Patrick Monnerat
8ef147c436
ftp,imap,pop3,smtp: reject STARTTLS server response pipelining
If a server pipelines future responses within the STARTTLS response, the
former are preserved in the pingpong cache across TLS negotiation and
used as responses to the encrypted commands.

This fix detects pipelined STARTTLS responses and rejects them with an
error.

CVE-2021-22947

Bug: https://curl.se/docs/CVE-2021-22947.html
2021-09-13 16:51:31 +02:00
Patrick Monnerat
364f174724
ftp,imap,pop3: do not ignore --ssl-reqd
In imap and pop3, check if TLS is required even when capabilities
request has failed.

In ftp, ignore preauthentication (230 status of server greeting) if TLS
is required.

Bug: https://curl.se/docs/CVE-2021-22946.html

CVE-2021-22946
2021-09-13 16:51:31 +02:00
Daniel Stenberg
4a8fa48946
http_proxy: fix the User-Agent inclusion in CONNECT
It should not refer to the uagent string that is allocated and created
for the end server http request, as that pointer may be cleared on
subsequent CONNECT requests.

Added test case 1184 to verify.

Reported-by: T200proX7 on github
Fixes #7705
Closes #7707
2021-09-13 12:29:59 +02:00
Marc Hoersken
60efeb1e0d
runtests: add option -u to error on server unexpectedly alive
Let's try to actually handle the server unexpectedly alive
case by first making them visible on CI builds as failures.

This is needed to detect issues with killing of the test
servers completely including nested process chains with
multiple PIDs per test server (including bash and perl).

On Windows/cygwin platforms this is especially helpful with
debugging PID mixups due to cygwin using its own PID space.

Reviewed-by: Daniel Stenberg
Closes #7180
2021-09-09 21:28:26 +02:00
Daniel Stenberg
d9b2d1d90f
copyrights: update copyright year ranges 2021-09-08 09:36:49 +02:00
Daniel Stenberg
eb2a5171da
curl: stop retry if Retry-After: is longer than allowed
If Retry-After: specifies a period that is longer than what fits within
--retry-max-time, then stop retrying immediately.

Added test 366 to verify.

Reported-by: Kari Pahula
Fixes #7675
Closes #7676
2021-09-06 14:46:45 +02:00
Daniel Stenberg
8fbdf8331e
test1173: check references to libcurl options
... that they refer to actual existing libcurl options.

Reviewed-by: Daniel Gustafsson
Closes #7656
2021-09-04 11:27:56 +02:00
Daniel Stenberg
12a2ed970f
opt-docs: verify man page sections + order
In every libcurl option man page there are now 8 mandatory sections that
must use the right name in the correct order and test 1173 verifies
this. Only 14 man pages needed adjustments.

The sections and the order is as follows:

 - NAME
 - SYNOPSIS
 - DESCRIPTION
 - PROTOCOLS
 - EXAMPLE
 - AVAILABILITY
 - RETURN VALUE
 - SEE ALSO

Reviewed-by: Daniel Gustafsson
Closes #7656
2021-09-04 11:27:46 +02:00
Daniel Stenberg
1731a77989
opt-docs: make sure all man pages have examples
Extended manpage-syntax.pl (run by test 1173) to check that every man
page for a libcurl option has an EXAMPLE section that is more than two
lines. Then fixed all errors it found and added examples.

Reviewed-by: Daniel Gustafsson
Closes #7656
2021-09-04 11:27:04 +02:00
Daniel Stenberg
424051d78b
connect: get local port + ip also when reusing connections
Regression. In d6a37c23a3 (7.75.0) we removed the duplicated storage
(connection + easy handle), so this info needs be extracted again even
for re-used connections.

Add test 435 to verify

Reported-by: Max Dymond
Fixes #7660
Closes #7662
2021-09-02 23:42:18 +02:00
Daniel Stenberg
3f4b1b4ee5
tests: adjust the tftpd output to work with hyper mode
By making them look less like http headers, the hyper mode "tweak"
doesn't interfere.

Enable test 2002 and 2003 in hyper builds (and 1280 which is unrelated
but should be enabled).

Closes #7658
2021-09-01 23:40:28 +02:00
Daniel Stenberg
e8c8775eaa
test365: verify response with chunked AND Content-Length headers 2021-08-31 10:04:03 +02:00
Daniel Stenberg
8a16e54c0c
http: ignore content-length if any transfer-encoding is used
Fixes #7643
Closes #7649
2021-08-31 10:04:03 +02:00
Daniel Stenberg
5dc594e44f
http: disallow >3-digit response codes
Make the built-in HTTP parser behave similar to hyper and reject any
HTTP response using more than 3 digits for the response code.

Updated test 1432 accordingly.
Enabled test 1432 in the hyper builds.

Closes #7641
2021-08-26 22:43:38 +02:00
Daniel Stenberg
9df52bf17a
test1280: CRLFify the response to please hyper
Closes #7639
2021-08-26 16:17:10 +02:00
Daniel Stenberg
9a30ac0957
tests: enable test 1129 for hyper builds
Closes #7638
2021-08-26 16:15:48 +02:00
Daniel Stenberg
e5d77dc2ca
c-hyper: handle HTTP/1.1 => HTTP/1.0 downgrade on reused connection
Enable test 1074

Closes #7617
2021-08-23 16:12:33 +02:00
Daniel Stenberg
38941ad6f3
c-hyper: deal with Expect: 100-continue combined with POSTFIELDS
Enable test 1130 and 1131

Closes #7616
2021-08-23 16:10:15 +02:00
a1346054
ec91b5a690
tests: be explicit about using 'python3' instead of 'python'
This fixes running tests in virtualenvs (or on distros) that no longer
have a symlink from python to python2 or python3.

Closes #7602
2021-08-23 15:59:43 +02:00
Daniel Stenberg
473b893c7e
DISABLED: enable 11 more tests for hyper builds
Closes #7612
2021-08-23 11:24:25 +02:00
Daniel Stenberg
46d4373ea7
setopt: enable CURLOPT_IGNORE_CONTENT_LENGTH for hyper
Since this option is also used for FTP, it needs to work to set for
applications even if hyper doesn't support it for HTTP. Verified by test
1137.

Updated docs to specify that the option doesn't work for HTTP when using
the hyper backend.

Closes #7614
2021-08-23 11:23:17 +02:00
Daniel Stenberg
0854781c18
test1138: remove trailing space to make work with hyper
Closes #7613
2021-08-23 11:21:38 +02:00
Daniel Stenberg
4e53b9430c
man pages: remove trailing whitespaces
Extended test 1173 (via the manpage-syntax.pl script) to detect and warn
for them.

Ref: #7602
Reported-by: a1346054 on github
Closes #7604
2021-08-21 16:44:03 +02:00
Daniel Stenberg
059379d20c
http_proxy: fix user-agent and custom headers for CONNECT with hyper
Enable test 287

Closes #7598
2021-08-20 13:51:06 +02:00
Daniel Stenberg
f46b83fc94
c-hyper: initial support for "dumping" 1xx HTTP responses
With the use hyper_request_on_informational()

Enable test 155 and 158

Closes #7597
2021-08-20 13:37:43 +02:00
Marc Hoersken
5b1c2dd1db
tests/*server.pl: flush output before executing subprocess
Also avoid shell processes staying around by using exec.
This is necessary to avoid output data being buffering
inside the process chain of Perl, Bash/Shell and our
test server binaries. On non-Windows systems the exec
will also make the subprocess replace the intermediate
shell, but on Windows it will at least bind the processes
together since there is no real fork or exec available.

See: https://cygwin.com/cygwin-ug-net/highlights.html
and: https://docs.microsoft.com/cpp/c-runtime-library/exec-wexec-functions
Ref: https://github.com/curl/curl/pull/7530#issuecomment-900949010

Reviewed-by: Daniel Stenberg
Reviewed-by: Jay Satiro
Closes #7530
2021-08-18 20:59:36 +02:00
Marc Hoersken
b5abbc3d57
tests/server/*.c: align handling of portfile argument and file
1. Call the internal variable portname (like pidname) everywhere.
2. Have a variable wroteportfile (like wrotepidfile) everywhere.
3. Make sure the file is cleaned up on exit (like pidfile).
4. Add parameter --portfile to usage outputs everywhere.

Reviewed-by: Daniel Stenberg

Replaces #7523
Closes #7574
2021-08-17 21:43:31 +02:00
Daniel Stenberg
8c9722bb1c
misc: update incorrect copyright year ranges
Closes #7577
2021-08-16 11:21:47 +02:00
Daniel Stenberg
32f6812b5a
c-hyper: initial step for 100-continue support
Enabled test 154

Closes #7568
2021-08-13 23:18:21 +02:00
Daniel Stenberg
c495dcd02e
http: consider cookies over localhost to be secure
Updated test31.
Added test 392 to verify secure cookies used for http://localhost

Reviewed-by: Daniel Gustafsson
Fixes #6733
Closes #7263
2021-08-10 11:20:21 +02:00
Jay Satiro
26f93cf038 test1565: fix windows build errors
- Use our wait_ms() instead of sleep() since Windows doesn't have the
  latter.

- Use a separate variable to keep track of whether the pthread_t thread
  id is valid.

On Windows pthread_t is not an integer type. pthread offers no macro for
invalid pthread_t thread id, so validity is kept track of separately.

Closes https://github.com/curl/curl/pull/7527
2021-08-10 03:29:40 -04:00
Bin Lan
5b9fedb461
tool/tests: fix potential year 2038 issues
The length of 'long' in a 32-bit system is 32 bits, which cannot be used
to save timestamps after 2038. Most operating systems have extended
time_t to 64 bits.

Remove the castings to long.

Closes #7466
2021-07-30 16:06:37 +02:00
Bernhard M. Wiedemann
8d0c1137c1
tests: make three tests pass until 2037
after 2038 something in test1915 fails on 32-bit OSes

Closes #7512
2021-07-30 12:39:34 +02:00
Marc Hoersken
c34bd93f60
tests/*server.py: remove pidfile on server termination
Avoid pidfile leaking/laying around after server already exited.

Reviewed-by: Daniel Stenberg
Closes #7506
2021-07-29 08:08:07 +02:00
Marc Hoersken
4a7bf79fcc
tests/servers: remove obsolete pid variable
Variable is not used since pidfile handling moved to util.[ch]

Reviewed-by: Jay Satiro
Closes #7482
2021-07-25 20:00:18 +02:00
Marc Hoersken
d9d26a6b2c
tests/servers: use our platform-aware pid for server verification
The pid used for server verification is later stored as pid2 in
the hash of running test servers and therefore used for shutdown.

The pid used for shutdown must be the platform-aware (Win32) pid
to avoid leaking test servers while running them using Cygwin/msys.

Reviewed-by: Jay Satiro
Closes #7481
2021-07-25 19:59:58 +02:00
Marc Hoersken
893f39c596
tests/runtests.pl: cleanup copy&paste mistakes and unused code
Reviewed-by: Jay Satiro
Part of #7481
2021-07-25 19:59:14 +02:00
Josh Soref
de1004eb0f
cleanup: spell DoH with a lowercase o
Signed-off-by: Josh Soref <jsoref@users.noreply.github.com>

Closes #7413
2021-07-16 23:53:13 +02:00
Daniel Stenberg
9053dbbf62
msnprintf: return number of printed characters excluding null byte
... even when the output is "capped" by the maximum length argument.

Clarified in the docs.

Closes #7361
2021-07-08 10:05:39 +02:00
Daniel Stenberg
e7416cfd2b
infof: remove newline from format strings, always append it
- the data needs to be "line-based" anyway since it's also passed to the
  debug callback/application

- it makes infof() work like failf() and consistency is good

- there's an assert that triggers on newlines in the format string

- Also removes a few instances of "..."

- Removes the code that would append "..." to the end of the data *iff*
  it was truncated in infof()

Closes #7357
2021-07-07 22:54:01 +02:00
Daniel Stenberg
c3d612662d
test1147: hyper doesn't allow "crazy" request headers like built-in
... so strip that from the test.

Closes #7349
2021-07-05 22:51:13 +02:00
Daniel Stenberg
a62ece3f57
c-hyper: bail on too long response headers
To match with built-in behaviors. Makes test 1154 work.

Closes #7350
2021-07-05 22:49:12 +02:00
Daniel Stenberg
0965348cd8
test1151: added missing CRLF to work with hyper
Closes #7350
2021-07-05 22:49:02 +02:00
Daniel Stenberg
e41feede34
test1116: hyper doesn't pass through "surprise-trailers"
Closes #7344
2021-07-05 22:39:42 +02:00
Daniel Stenberg
07fa74d34a
test1519: adjusted to work with hyper
Closes #7333
2021-07-01 14:26:06 +02:00
Daniel Stenberg
a80f8ea182
test1518: adjusted to work with hyper
... by making sure the stdout output doesn't look like HTTP headers.

Closes #7333
2021-07-01 14:25:57 +02:00
Daniel Stenberg
7f11282d5f
test1514: add a CRLF to the response to make it correct
Makes hyper accept it fine instead returning HYPERE_UNEXPECTED_EOF on
us.

Closes #7334
2021-07-01 14:19:55 +02:00
Daniel Stenberg
ca60a4398f
lib677: make it survive torture testing
Follow-up to a5ab72d5ed

Closes #7300
2021-06-27 23:20:18 +02:00
Daniel Stenberg
a5ab72d5ed
test677: IMAP CONNECT_ONLY, custom command and then exit
Adjusted ftpserver.pl to add support for the IMAP IDLE command

Adjusted test 660 to sync with the fix
2021-06-24 09:07:40 +02:00
Li Xinwei
30e491e5c9
cmake: fix support for UnixSockets feature on Win32
Move the definition of sockaddr_un struct from config-win32.h to
curl_setup.h, so that it could be shared by all build systems.

Add ADDRESS_FAMILY typedef for old mingw, now old mingw can also use
unix sockets.

Also fix the build of tests/server/sws.c on Win32 when USE_UNIX_SOCKETS
is defined.

Closes #7034
2021-06-21 14:52:27 +02:00
Daniel Stenberg
80e1054fe5
hyper: propagate errors back up from read callbacks
Makes test 513 work with hyper

Closes #7266
2021-06-17 08:19:31 +02:00
Daniel Stenberg
b67d3ba73e
curl_url_set: reject spaces in URLs w/o CURLU_ALLOW_SPACE
They were never officially allowed and slipped in only due to sloppy
parsing. Spaces (ascii 32) should be correctly encoded (to %20) before
being part of a URL.

The new flag bit CURLU_ALLOW_SPACE when a full URL is set, makes libcurl
allow spaces.

Updated test 1560 to verify.

Closes #7073
2021-06-15 10:49:49 +02:00
Daniel Stenberg
bbbc5de93f
netrc: skip 'macdef' definitions
Add test 494 to verify

Reported-by: Harry Sintonen
Fixes #7238
Closes #7244
2021-06-13 23:44:41 +02:00
Gealber Morales
791937b881
mqtt: add support for username and password
Minor-edits-by: Daniel Stenberg
Added test 2200 to 2205

Closes #7243
2021-06-12 23:50:13 +02:00
Daniel Stenberg
f60f51721c
runtests: init $VERSION to avoid warnings when using -l 2021-06-11 14:26:32 +02:00
Daniel Stenberg
a15d146f47
tests: update README about servers and port numbers
Closes #7242
2021-06-11 12:48:56 +02:00
Christian Weisgerber
02dfe71937
sws: malloc request struct instead of using stack
... 2MB requests is otherwise just too big for some systems.

(The allocations are not freed properly.)

Bug: https://curl.se/mail/lib-2021-06/0018.html

Closes #7235
2021-06-11 11:13:24 +02:00
yb999
b9cdcc7a17
tests: update README.md with a missing single quote
Closes #7231
2021-06-10 16:11:03 +02:00
Daniel Stenberg
7cfd8f037a
tests/data/DISABLED: add tests not working with hyper
The goal is to remove them all from here over time.

Closes #7209
2021-06-10 08:42:55 +02:00
Daniel Stenberg
dd47b0cce9
runtests: also find the last test in Makefile.inc
Closes #7209
2021-06-10 08:42:52 +02:00
Daniel Stenberg
77ae8f26ad
test3010: work with hyper mode
Closes #7209
2021-06-10 08:42:50 +02:00
Daniel Stenberg
75391045cf
test1594/1595/1596: fix to work in hyper mode
Closes #7209
2021-06-10 08:42:44 +02:00
Daniel Stenberg
30e50de53e
test1438/1457: add HTTP keyword to make hyper mode work
Closes #7209
2021-06-10 08:42:41 +02:00
Daniel Stenberg
5f70cb6461
test1340/1341: adjusted for hyper mode
Closes #7209
2021-06-10 08:42:38 +02:00
Daniel Stenberg
769d9176e3
test1218: adjusted for hyper mode
Closes #7209
2021-06-10 08:42:35 +02:00
Daniel Stenberg
6101ddc24e
test1216: adjusted for hyper mode
Closes #7209
2021-06-10 08:42:33 +02:00
Daniel Stenberg
340d6b4589
test1230: adjust to work in hyper mode
Closes #7209
2021-06-10 08:42:30 +02:00
Daniel Stenberg
c214a6a17b
c-hyper: abort CONNECT response reading early on non 2xx responses
Fixes test 493

Closes #7209
2021-06-10 08:42:27 +02:00
Daniel Stenberg
aeb064db03
test434: add HTTP keyword
Closes #7209
2021-06-10 08:42:23 +02:00
Daniel Stenberg
e7ae6f013d
test599: adjusted to work in hyper mode
Closes #7209
2021-06-10 08:42:19 +02:00
Daniel Stenberg
cd5ca9bb38
test566: adjust to work with hyper mode
Closes #7209
2021-06-10 08:42:13 +02:00
Daniel Stenberg
76035e612a
runtests: skip disabled tests unless -f is used
To make it easier to write ranges like '115 to 229' without that
explicitly enabling tests that are listed in DISABLED, this makes
runtests always skip disabled tests unless the -f command line option is
used.

Previously the code attempted to not run such tests, but didn't do it
correctly.

Closes #7212
2021-06-08 23:35:37 +02:00
Daniel Stenberg
3ac9b80525
test644: remove as duplicate of test 587
Closes #7208
2021-06-08 16:07:56 +02:00
Daniel Stenberg
48b0cca65e
test500: adjust to work with hyper mode 2021-06-07 23:54:54 +02:00
Daniel Stenberg
57e46dce58
test433: adjust for hyper mode
Closes #7205
2021-06-07 23:14:09 +02:00
Daniel Stenberg
528f22736d
test395: hyper cannot work around > 64 bit content-lengths like built-in
Closes #7205
2021-06-07 23:14:06 +02:00
Daniel Stenberg
77590673e9
test394: hyper returns a different error
Closes #7205
2021-06-07 23:14:03 +02:00
Daniel Stenberg
1449713dd3
test393: make Content-Length fit within 64 bit for hyper
Closes #7205
2021-06-07 23:14:00 +02:00
Daniel Stenberg
ae37f46e80
test347: CRLFify to work in hyper mode
Closes #7205
2021-06-07 23:13:55 +02:00
Daniel Stenberg
71da11a49d
test339: CRLFify better to work in hyper mode
Closes #7205
2021-06-07 23:13:47 +02:00
Daniel Stenberg
b26f1dc14f
test328: avoid a header-looking body to make hyper mode work
The test still works the same, just modified two bytes in the content.

Closes #7203
2021-06-07 13:26:27 +02:00
Daniel Stenberg
265b14d6b3
metalink: remove
Warning: this will make existing curl command lines that use metalink to
stop working.

Reasons for removal:

1. We've found several security problems and issues involving the
   metalink support in curl. The issues are not detailed here. When
   working on those, it become apparent to the team that several of the
   problems are due to the system design, metalink library API and what
   the metalink RFC says. They are very hard to fix on the curl side
   only.

2. The metalink usage with curl was only very briefly documented and was
   not following the "normal" curl usage pattern in several ways, making
   it surprising and non-intuitive which could lead to further security
   issues.

3. The metalink library was last updated 6 years ago and wasn't so
   active the years before that either. An unmaintained library means
   there's a security problem waiting to happen. This is probably reason
   enough.

4. Metalink requires an XML parsing library, which is complex code (even
   the smaller alternatives) and to this day often gets security
   updates.

5. Metalink is not a widely used curl feature. In the 2020 curl user
   survey, only 1.4% of the responders said that they'd are using it. In
   2021 that number was 1.2%. Searching the web also show very few
   traces of it being used, even with other tools.

6. The torrent format and associated technology clearly won for
   downloading large files from multiple sources in parallel.

Cloes #7176
2021-06-07 08:14:25 +02:00
Daniel Stenberg
510e6e9a19
tests/data/Makefile.inc: error: trailing backslash on last line
Follow-up to d8dcb399b8
2021-06-04 16:52:34 +02:00
Alexis Vachette
d8dcb399b8
url: bad CURLOPT_CONNECT_TO syntax now returns error
Added test 3020 to verify

Closes #7183
2021-06-04 14:08:30 +02:00
Daniel Stenberg
628ebd82b9
test269: disable for hyper
--ignore-content-length / CURLOPT_IGNORE_CONTENT_LENGTH doesn't work
with hyper.

Closes #7184
2021-06-03 23:11:11 +02:00
Daniel Stenberg
b193fd3bbd
runtests: enable 'hyper mode' only for HTTP tests
The 'hyper mode' makes line-ending checks work in the test suite for
when hyper is used. Now it also requires that HTTP or HTTPS are
mentioned as keywords to be enabled so that it doesn't wrongly adjusts
tests for other protocols.

This makes test 271 (TFTP) work again in hyper enabled builds.

Closes #7185
2021-06-03 23:08:47 +02:00
Alexis Vachette
cb39b4b7c4
hostip: bad CURLOPT_RESOLVE syntax now returns error
Added test 3019
Fixes #7170
Closes #7174
2021-06-03 13:53:18 +02:00
Daniel Stenberg
3c0f4622cd
runtests: parse data/Makefile.inc instead of using make
The warning about missing entries in that file then doesn't require that
the Makefile has been regenerated which was confusing.

The scan for the test num is a little more error prone than before
(since now it doesn't actually verify that it is legitimate Makefile
syntax), but I think it is good enough.

Closes #7177
2021-06-02 23:21:18 +02:00
Harry Sintonen
e4db6fb02c
filecheck: quietly remove test-place/*~
Closes #7179
2021-06-02 23:17:44 +02:00
Daniel Stenberg
6793332eba
CURLE_SETOPT_OPTION_SYNTAX: new error name for wrong setopt syntax
For options that pass in lists or strings that are subsequently parsed
and must be correct. This broadens the scope for the option previously
known as CURLE_TELNET_OPTION_SYNTAX but the old name is of course still
provided as a #define for existing applications.

Closes #7175
2021-06-02 23:03:55 +02:00
Daniel Stenberg
9dc0baf5c6
tests: fix Accept-Encoding strips to work with Hyper builds
The previous strip also removed the CR which turned problematic.

valgrind.supp: add zstd suppression using hyper

Reported-and-analyzed-by: Kevin Burke
Fixes #7169
Closes #7171
2021-06-02 22:59:33 +02:00
Harry Sintonen
8ccf75532b
mqtt: detect illegal and too large file size
Add test 3017 and 3018 to verify.
Closes #7166
2021-06-02 13:34:17 +02:00
Daniel Stenberg
9097843e8f
test178: adjust for hyper
Hyper returns the same error for wrong HTTP version as for negative
content-length. Test 178 verifies that negative content-length is
rejected but the hyper backend will return a different error for it (and
without any helpful message telling why the message was bad). It will
also not return any headers at all for the response, not even the ones
that arrived before the error.

Closes #7147
2021-05-29 23:16:04 +02:00
Viktor Szakats
3976dd67a2
docs: use --max-redirs instead of --max-redir
For consistency.

Closes #7130
2021-05-26 14:49:52 +02:00
Daniel Stenberg
82d334001f
copyright: update copyright year ranges to 2021 2021-05-26 08:18:11 +02:00
Jacob Hoffman-Andrews
a62e6435f4
rustls: switch read_tls and write_tls to callbacks
And update to 0.6.0, including a rename from session to connection for
many fields.

Closes #7071
2021-05-24 16:40:59 +02:00
Daniel Stenberg
b03b82a85f
docs/tests: remove freenode references 2021-05-24 00:21:00 +02:00
Lucas Clemente Vella
84d2839740
CURLOPT_IPRESOLVE: preventing wrong IP version from being used
In some situations, it was possible that a transfer was setup to
use an specific IP version, but due do DNS caching or connection
reuse, it ended up using a different IP version from requested.

This commit changes the effect of CURLOPT_IPRESOLVE from simply
restricting address resolution to preventing the wrong connection
type being used, when choosing a connection from the pool, and
to restricting what addresses could be used when establishing
a new connection.

It is important that all addresses versions are resolved, even if
not used in that transfer in particular, because the result is
cached, and could be useful for a different transfer with a
different CURLOPT_IPRESOLVE setting.

Closes #6853
2021-05-20 16:58:31 +02:00
Daniel Stenberg
d9eb3e316f
test2100: make it run with and require IPv6
Closes #7083
2021-05-20 08:32:43 +02:00
Daniel Stenberg
1f90d08371
tests/getpart: generate output URL encoded for better diffs
Closes #7083
2021-05-20 08:32:38 +02:00
Daniel Stenberg
030d539164
sws: allow HTTP requests up to 2MB in size
To allow tests with slightly larger payloads. Like #7071 ...

Closes #7075
2021-05-16 12:56:34 +02:00
Daniel Stenberg
e35d7c0492
mem-include-scan.pl: require a non-word letter before memory funcs
... so that ldap_memfree() for example doesn't match the scan for free.

Closes #7061
2021-05-15 17:47:37 +02:00
Dan Fandrich
56cf2de5ac tests: Fix some tag matching issues in a number of tests 2021-05-12 01:17:16 -07:00
Daniel Stenberg
51c0ebcff2
http: deal with partial CONNECT sends
Also added 'CURL_SMALLSENDS' to make Curl_write() send short packets,
which helped verifying this even more.

Add test 363 to verify.

Reported-by: ustcqidi on github
Fixes #6950
Closes #7024
2021-05-08 10:49:16 +02:00
Marc Hoersken
d698c70421
sockfilt: fix invalid increment of handles index variable nfd
Only increment the array index if we actually stored a handle.

Follow up to e917492048
Closes #6992
2021-05-05 22:30:48 +02:00
Marc Hoersken
b9239e0f9c
sockfilt: avoid getting stuck waiting for writable socket
Reset FD_WRITE event using the same approach as in multi.c

Follow up to b36442b243
Closes #6992
2021-05-05 22:30:12 +02:00
Jay Satiro
5a1ec19fda test678: Fix for Windows multibyte builds
Follow-up to 77fc385 from yesterday.

Bug: https://github.com/curl/curl/pull/6662#issuecomment-832966557
Reported-by: Marc Hörsken
2021-05-05 16:25:45 -04:00
Gilles Vollant
77fc3859b2 SSL: support in-memory CA certs for some backends
- New options CURLOPT_CAINFO_BLOB and CURLOPT_PROXY_CAINFO_BLOB to
  specify in-memory PEM certificates for OpenSSL, Schannel (Windows)
  and Secure Transport (Apple) SSL backends.

Prior to this change PEM certificates could only be imported from a file
and not from memory.

Co-authored-by: moparisthebest@users.noreply.github.com

Ref: https://github.com/curl/curl/pull/4679
Ref: https://github.com/curl/curl/pull/5677
Ref: https://github.com/curl/curl/pull/6109

Closes https://github.com/curl/curl/pull/6662
2021-05-05 02:29:16 -04:00
David Cook
70cf50fb4a
tests: ignore case of chunked hex numbers in tests
When hyper is used, it emits uppercase hexadecimal numbers for chunked
encoding lengths. Without hyper, lowercase hexadecimal numbers are used.
This change adds preprocessor statements to tests where this is an
issue, and adapts the fixtures to match.

Closes #6987
2021-05-04 15:57:57 +02:00
Daniel Stenberg
91d9fead66
libtest: remove lib530.c
Follow up from e50a877df when test 530 was removed. Since then this
source file has not been used/needed.

Closes #6999
2021-05-04 14:45:52 +02:00
Daniel Stenberg
23e9b96a64
FILEFORMAT: mention sectransp as a feature
Been supported since at least 40259ca65

Closes #7001
2021-05-04 14:45:01 +02:00
ejanchivdorj
94241a9e78
CURLcode: add CURLE_SSL_CLIENTCERT
When a TLS server requests a client certificate during handshake and
none can be provided, libcurl now returns this new error code
CURLE_SSL_CLIENTCERT

Only supported by Secure Transport and OpenSSL for TLS 1.3 so far.

Closes #6721
2021-05-03 17:11:01 +02:00
Daniel Stenberg
4708f29f1c
tests/disable-scan.pl: also scan all m4 files
Fixes test 1165 when functions are moved from configure.ac to files in
m4/
2021-04-22 23:19:47 +02:00
Daniel Stenberg
063d3f3b96
tidy-up: make conditional checks more consistent
... remove '== NULL' and '!= 0'

Closes #6912
2021-04-22 09:10:17 +02:00
Patrick Monnerat
34cf40321c
bufref: buffer reference support
A struct bufref holds a buffer pointer, a data size and a destructor.
When freed or its contents are changed, the previous buffer is implicitly
released by the associated destructor. The data size, although not used
internally, allows binary data support.

A unit test checks its handling methods: test 1661

Closes #6654
2021-04-22 09:05:53 +02:00
Patrick Monnerat
3fb6e5a010
test server: take care of siginterrupt() deprecation
Closes #6529
2021-04-22 08:32:43 +02:00
Marc Hoersken
2df64390b3
lib1564.c: enable last wakeup test part on Windows
Suggested-by: Gergely Nagy
Reviewed-by: Jay Satiro
Reviewed-by: Marcel Raad

Closes #6245
2021-04-21 20:32:33 +02:00
Daniel Stenberg
3057c6cd73
Revert "cmake: make libcurl library output name configurable"
This reverts commit 1cba36d216.

CMake provides properties that can be set on a target to rename the
output artifact without changing the name of a target.

Ref: #6899
2021-04-21 15:03:26 +02:00
Ralph Langendam
1cba36d216
cmake: make libcurl library output name configurable
Closes #6899
2021-04-21 10:03:07 +02:00
Daniel Stenberg
415d6fa7da
sws: #ifdef S_IFSOCK use
SCO OpenServer 5.0.7 does not define S_IFSOCK.

Reported-by: Kevin R. Bulgrien
Bug: https://curl.se/mail/lib-2021-04/0074.html
Closes #6926
2021-04-21 09:35:31 +02:00
Daniel Stenberg
04488851e2
urlapi: make sure no +/- signs are accepted in IPv4 numericals
Follow-up to 56a037cc0a. Extends test 1560 to verify.

Reported-by: Tuomas Siipola
Fixes #6916
Closes #6917
2021-04-21 09:17:55 +02:00
Daniel Stenberg
6b97f1311a
test1660: check the created HSTS file as text mode
Closes #6922
2021-04-21 08:18:52 +02:00
Daniel Stenberg
7569518e92
test 493: require https in curl to run
Closes #6927
2021-04-20 10:01:19 +02:00
Daniel Stenberg
56a037cc0a
urlapi: "normalize" numerical IPv4 host names
When the host name in a URL is given as an IPv4 numerical address, the
address can be specified with dotted numericals in four different ways:
a32, a.b24, a.b.c16 or a.b.c.d and each part can be specified in
decimal, octal (0-prefixed) or hexadecimal (0x-prefixed).

Instead of passing on the name as-is and leaving the handling to the
underlying name functions, which made them not work with c-ares but work
with getaddrinfo, this change now makes the curl URL API itself detect
and "normalize" host names specified as IPv4 numericals.

The WHATWG URL Spec says this is an okay way to specify a host name in a
URL. RFC 3896 does not allow them, but curl didn't prevent them before
and it seems other RFC 3896-using tools have not either. Host names used
like this are widely supported by other tools as well due to the
handling being done by getaddrinfo and friends.

I decided to add the functionality into the URL API itself so that all
users of these functions get the benefits, when for example wanting to
compare two URLs. Also, it makes curl built to use c-ares now support
them as well and make curl builds more consistent.

The normalization makes HTTPS and virtual hosted HTTP work fine even
when curl gets the address specified using one of the "obscure" formats.

Test 1560 is extended to verify.

Fixes #6863
Closes #6871
2021-04-19 08:34:55 +02:00
Daniel Stenberg
d71ff2b9db
hsts: enable by default
No longer considered experimental.

Closes #6700
2021-04-19 08:22:16 +02:00
Daniel Stenberg
6fc805d0c1
test972: verify the json output with jsonlint
Make sure one of the azure jobs has jsonlint installed so that the test
runs there.

Ref: #6905
2021-04-17 11:03:40 +02:00
Jay Satiro
2f78be51eb
tool_writeout: fix the HTTP_CODE json output
Update test 970 accordingly.

Reported-by: Michal Rus
Fixes #6905
Closes #6906
2021-04-17 11:03:30 +02:00
Victor Vieux
0d7c55bd57
tool_getparam: replace (in-place) '%20' by '+' according to RFC1866
Signed-off-by: Victor Vieux <victorvieux@gmail.com>

Closes #6895
2021-04-15 13:45:12 +02:00
Daniel Stenberg
7bdec2a08b
configure: provide --with-openssl, deprecate --with-ssl
Makes the option more explicit.

Closes #6887
2021-04-15 09:08:34 +02:00
Daniel Stenberg
520bd5225c
cookie: CURLOPT_COOKIEFILE set to NULL switches off cookies
Add test 676 to verify that setting CURLOPT_COOKIEFILE to NULL again clears
the cookiejar from memory.

Reported-by: Stefan Karpinski
Fixes #6889
Closes #6891
2021-04-14 23:09:36 +02:00
Patrick Monnerat
9c1e1a6105
ntlm: fix negotiated flags usage
According to Microsoft document MS-NLMP, current flags usage is not
accurate: flag NTLMFLAG_NEGOTIATE_NTLM2_KEY controls the use of
extended security in an NTLM authentication message and NTLM version 2
cannot be negotiated within the protocol.

The solution implemented here is: if the extended security flag is set,
prefer using NTLM version 2 (as a server featuring extended security
should also support version 2). If version 2 has been disabled at
compile time, use extended security.

Tests involving NTLM are adjusted to this new behavior.

Fixes #6813
Closes #6849
2021-04-09 09:40:56 +02:00
Michael Forney
812fce9dcf
configure: remove use of RETSIGTYPE
This was previously defined by the obsolete AC_TYPE_SIGNAL macro,
which was removed in 2682e5f5. The deprecation text says

> Your code may safely assume C89 semantics that RETSIGTYPE is void.

So, remove it and just use void instead.

Closes #6861
2021-04-07 16:01:05 +02:00
Daniel Stenberg
605aa03ac1
http_proxy: only loop on 407 + close if we have credentials
... to fix the retry-loop.

Add test 718 to verify.

Reported-by: Daniel Kurečka
Fixes #6828
Closes #6850
2021-04-05 22:49:07 +02:00
Luke Granger-Brown
6d930d7306
file: support GETing directories again
After 957bc1881e, we no longer compute an
expected_size for directories. This has the upshot that when we compare
even an empty Range with the available size, we fail.

This brings back the previous behaviour, which was to succeed, but with
empty content. This also removes the "Accept-ranges: bytes" header,
which is nonsensical on directories.

Adds test 3016
Fixes #6845
Closes #6846
2021-04-05 12:22:07 +02:00
Daniel Stenberg
33ba0ecf48
define: remove CURL_DISABLE_NTLM ifdefs
It was never defined anywhere. Fixed disable-scan (test 1165) to also
scan headers, which found this issue.

Closes #6809
2021-03-29 14:43:54 +02:00
Viktor Szakats
7214288898
transfer: strip credentials from the auto-referer header field
Added test 2081 to verify.

CVE-2021-22876

Bug: https://curl.se/docs/CVE-2021-22876.html
2021-03-28 23:19:55 +02:00
Daniel Stenberg
85e6975643
copyright: update copyright year ranges to 2021
Reviewed-by: Emil Engler
Closes #6802
2021-03-27 23:00:14 +01:00
Daniel Stenberg
d003b0213a
tests/README.md: mentioned that en_US.UTF-8 is required
Reported-by: Oumph on github
Fixes #6768
2021-03-26 17:54:12 +01:00
Daniel Stenberg
3bbf62b5a4
http: strip default port from URL sent to proxy
To make sure the Host: header and the URL provide the same authority
portion when sent to the proxy, strip the default port number from the
URL if one was provided.

Reported-by: Michael Brown
Fixes #6769
Closes #6778
2021-03-23 13:33:49 +01:00
Daniel Stenberg
bd738bef12
ftpserver: only load the preprocessed test file
We always preprocess and tests are no longer sensible to load "raw"

Closes #6738
2021-03-19 15:57:27 +01:00
Daniel Stenberg
e2b4df7b5e
tests: use %TESTNUMBER instead of fixed number
This makes the tests easier to copy and relocate to other test numbers
without having to update content.

Closes #6738
2021-03-19 15:57:21 +01:00
Daniel Stenberg
6d176bee59
http: make 416 not fail with resume + CURLOPT_FAILONERRROR
When asked to resume a download, libcurl will convert that to HTTP logic
and if then the entire file is already transferred it will result in a
416 response from the HTTP server. With CURLOPT_FAILONERRROR set in that
scenario, it should *not* lead to an error return.

Updated test 1156, added test 1273

Reported-by: Jonathan Watt
Fixes #6740
Closes #6753
2021-03-17 08:26:46 +01:00
Daniel Stenberg
f83d4ea56e
gen.pl: quote "bare" minuses in the nroff curl.1
Reported-by: Alejandro Colomar
Fixes #6698
Closes #6722
2021-03-14 23:44:43 +01:00
Daniel Stenberg
f073a427b4
test220/314: adjust to run with Hyper 2021-03-11 22:45:27 +01:00
Daniel Stenberg
25837e32b0
test306: make it not run with Hyper
... as it tests HTTP/0.9 which Hyper doesn't support.
2021-03-11 22:43:09 +01:00
Daniel Stenberg
366c0a4aee
test304: header CRLF cleanup to work with Hyper 2021-03-11 22:43:09 +01:00
Daniel Stenberg
7b2f0676c2
FTP: allow SIZE to fail when doing (resumed) upload
Added test 362 to verify.

Reported-by: Jordan Brown
Regression since 7ea2e1d0c5 (7.73.0)
Fixes #6715
Closes #6725
2021-03-11 16:13:11 +01:00
Daniel Stenberg
dcccd8dd25
configure: provide Largefile feature for curl-config
... as cmake now does it correctly, and make test1014 check for it

Closes #6702
2021-03-11 10:12:30 +01:00
kokke
4088b25b33
lib1536: check ptr against NULL before dereferencing it
Closes #6710
2021-03-10 14:03:57 +01:00
kokke
b2d906754c
lib1537: check ptr against NULL before dereferencing it
Fixes #6707
Closes #6708
2021-03-10 14:01:55 +01:00
Marc Hoersken
311c31ec8e
runtests.pl: kill processes locking test log files
Introduce a new runtests.pl command option: -rm

For now only required and implemented for Windows.
Ignore stunnel logs due to long running processes.

Requires Sysinternals handle[64].exe to be on PATH.

Reviewed-by: Jay Satiro

Ref: #6058
Closes #6179
2021-03-01 20:19:01 +01:00
Marc Hoersken
b2b3c91ed7
pathhelp.pm: fix use of pwd -L in Msys environment
While Msys2 has a pwd binary which supports -L,
Msys1 only has a shell built-in with that feature.

Reviewed-by: Jay Satiro

Part of #6179
2021-03-01 20:18:04 +01:00
Jay Satiro
5743d66b91 tool_help: Increase space between option and description
- Increase the minimum number of spaces between the option and the
  description from 1 to 2.

Before:
~~~
 -u, --user <user:password> Server user and password
 -A, --user-agent <name> Send User-Agent <name> to server
 -v, --verbose       Make the operation more talkative
 -V, --version       Show version number and quit
 -w, --write-out <format> Use output FORMAT after completion
     --xattr         Store metadata in extended file attributes
~~~

After:
~~~
 -u, --user <user:password>  Server user and password
 -A, --user-agent <name>  Send User-Agent <name> to server
 -v, --verbose       Make the operation more talkative
 -V, --version       Show version number and quit
 -w, --write-out <format>  Use output FORMAT after completion
     --xattr         Store metadata in extended file attributes
~~~

Closes https://github.com/curl/curl/pull/6674
2021-02-28 13:41:15 -05:00
Daniel Gustafsson
82c583dcf0 cookies: Support multiple -b parameters
Previously only a single -b cookie parameter was supported with the last
one winning.  This adds support for supplying multiple -b params to have
them serialized semicolon separated.  Both cookiefiles and cookies can be
entered multiple times.

Closes #6649
Reviewed-by: Daniel Stenberg <daniel@haxx.se>
2021-02-25 18:12:28 +01:00
Jay Satiro
3d954e4e26 test1188: Check for --fail HTTP status
- Change the test to check for curl error on HTTP 404 Not Found.

test1188 tests "--write-out with %{onerror} and %{urlnum} to stderr".
Prior to this change it did that by specifying a non-existent host which
would cause an error. ISPs may hijack DNS and resolve non-existent hosts
so the test would not work if that was the case.

Ref: https://en.wikipedia.org/wiki/DNS_hijacking#Manipulation_by_ISPs
Ref: https://github.com/curl/curl/issues/6621
Ref: https://github.com/curl/curl/pull/6623

Closes https://github.com/curl/curl/pull/6637
2021-02-20 14:41:13 -05:00
Viktor Szakats
44872aefc2
http: add support to read and store the referrer header
- add CURLINFO_REFERER libcurl option
- add --write-out '%{referer}' command-line option
- extend --xattr command-line option to fill user.xdg.referrer.url extended
  attribute with the referrer (if there was any)

Closes #6591
2021-02-19 13:57:19 +00:00
Daniel Stenberg
35ca04ce1b
test1188: change error from connect to resolve error
Using the %NOLISTENPORT to trigger a connection failure is somewhat
"risky" (since it isn't guaranteed to not be listened to) and caused
occasional CI problems. This fix changes the infused error to be a more
reliable one but still verifies the --write-out functionality properly -
which is the purpose of this test.

Reported-by: Jay Satiro
Fixes #6621
Closes #6623
2021-02-18 11:10:47 +01:00
Daniel Stenberg
46620b9743
http: use credentials from transfer, not connection
HTTP auth "accidentally" worked before this cleanup since the code would
always overwrite the connection credentials with the credentials from
the most recent transfer and since HTTP auth is typically done first
thing, this has not been an issue. It was still wrong and subject to
possible race conditions or future breakage if the sequence of functions
would change.

The data.set.str[] strings MUST remain unmodified exactly as set by the
user, and the credentials to use internally are instead set/updated in
state.aptr.*

Added test 675 to verify different credentials used in two requests done
over a reused HTTP connection, which previously behaved wrongly.

Fixes #6542
Closes #6545
2021-02-13 22:36:15 +01:00
Daniel Stenberg
e992770e8d
test433: clear some home dir env variables
Follow-up to bd6b54ba1f

... so that XDG_CONFIG_HOME is the only home dir variable set and thus
used correctly in the test!

Fixes #6599
Closes #6600
2021-02-12 14:02:07 +01:00
Daniel Stenberg
88dd1a8a11
urldata: don't touch data->set.httpversion at run-time
Rename it to 'httpwant' and make a cloned field in the state struct as
well for run-time updates.

Also: refuse non-supported HTTP versions. Verified with test 129.

Closes #6585
2021-02-12 08:13:37 +01:00
Viktor Szakats
bd6b54ba1f
tests: disable .curlrc in more environments
by also setting CURL_HOME and XDG_CONFIG_HOME envvars to the local
directory.

Reviewed-by: Daniel Stenberg
Fixes #6595
Closes #6596
2021-02-11 22:06:02 +00:00
Daniel Stenberg
8a964cb217
curl: add --fail-with-body
Prevent both --fail and --fail-with-body on the same command line.

Verify with test 349, 360 and 361.

Closes #6449
2021-02-11 08:35:27 +01:00
Jay Satiro
89e572af82 tests: Fix tests failing due to change in curl --help
Follow-up to parent 3183217 which added add missing <mode> argument to
--create-file-mode <mode>.

Ref: https://github.com/curl/curl/issues/6590
2021-02-10 15:57:26 -05:00
Marcel Raad
1636076e70
server: remove redundant condition
`end` is always non-null here.

Closes https://github.com/curl/curl/pull/6576
2021-02-10 16:47:34 +01:00
Marcel Raad
cc9c4e0f4e
mqttd: remove unused variable
Closes https://github.com/curl/curl/pull/6576
2021-02-10 16:47:21 +01:00
Marcel Raad
4c02d003ff
tests: reduce variable scopes
Closes https://github.com/curl/curl/pull/6576
2021-02-10 16:47:09 +01:00
Daniel Stenberg
528f71c2ec
ftp: add 'list_only' to the transfer state struct
and rename it from 'ftp_list_only' since it is also used for SSH and
POP3. The state is updated internally for 'type=D' FTP URLs.

Added test case 1570 to verify.

Closes #6578
2021-02-09 14:06:28 +01:00
Daniel Stenberg
115c9e27f5
ftp: add 'prefer_ascii' to the transfer state struct
... and make sure the code never updates 'set.prefer_ascii' as it breaks
handle reuse which should use the setting as the user specified it.

Added test 1569 to verify: it first makes an FTP transfer with ';type=A'
and then another without type on the same handle and the second should
then use binary. Previously, curl failed this.

Closes #6578
2021-02-09 14:06:28 +01:00
Jay Satiro
cb2dc1ba89 lib: use int type for more port variables
This is a follow-up to 764c6bd. Prior to that change port variables
were usually type long.

Closes https://github.com/curl/curl/pull/6553
2021-02-09 02:53:31 -05:00
Jay Satiro
65ca229461 tool_writeout: refactor write-out and write-out json
- Deduplicate the logic used by write-out and write-out json.

Rather than have separate writeLong, writeString, etc, logic for
each of write-out and write-out json instead have respective shared
functions that can output either format and a 'use_json' parameter to
indicate whether it is json that is output.

This will make it easier to maintain. Rather than have to go through
two sets of logic now we only have to go through one.

- Support write-out %{errormsg} and %{exitcode} in json.

- Clarify in the doc that %{exitcode} is the exit code of the transfer.

Prior to this change it just said "The numerical exitcode" which
implies it's the exit code of the tool, and it's not necessarily that.

Closes https://github.com/curl/curl/pull/6544
2021-02-09 02:48:27 -05:00
Stephan Szabo
1269c80af1 file: Support unicode urls on windows
Closes https://github.com/curl/curl/pull/6501
2021-02-09 02:45:51 -05:00
Fabian Keil
461e3ef680
tests/server: Bump MAX_TAG_LEN to 200
This is useful for tests containing HTML inside of <data> sections.
For <img> tags it's not uncommon to be longer than the previous
limit of 79 bytes.

An example of a previously problem-causing tag is:
<img src="http://config.privoxy.org/send-banner?type=auto" border="0" title="Killed-http://www.privoxy.org/images/privoxy.png-by-size" width="88" height="31">
which is needed for a Privoxy test for the banners-by-size filter.

Previously it caused server failures like:
12:29:05.786961 ====> Client connect
12:29:05.787116 accept_connection 3 returned 4
12:29:05.787194 accept_connection 3 returned 0
12:29:05.787285 Read 119 bytes
12:29:05.787345 Process 119 bytes request
12:29:05.787407 Got request: GET /banners-by-size/9 HTTP/1.1
12:29:05.787464 Requested test number 9 part 0
12:29:05.787686 getpart() failed with error: -2
12:29:05.787744 - request found to be complete (9)
12:29:05.787912 getpart() failed with error: -2
12:29:05.788048 Wrote request (119 bytes) input to log/server.input
12:29:05.788157 Send response test9 section <data>
12:29:05.788443 getpart() failed with error: -2
12:29:05.788498 instructed to close connection after server-reply
12:29:05.788550 ====> Client disconnect 0
12:29:05.871448 exit_signal_handler: 15
12:29:05.871714 signalled to die
12:29:05.872040 ========> IPv4 sws (port 21108 pid: 51758) exits with signal (15)
2021-02-09 08:16:40 +01:00
Fabian Keil
e11e5b2d46
tests/badsymbols.pl: when opening '$incdir' fails include it in the error message 2021-02-09 08:16:40 +01:00
Fabian Keil
e6f4ab3477
runtests.1: document -o, -P, -L, and -E 2021-02-09 08:16:40 +01:00
Fabian Keil
203dc00718
runtests.pl: add %TESTNUMBER variable to make copying tests more convenient 2021-02-09 08:16:40 +01:00
Fabian Keil
8ad31a8c6e
runtests.pl: add an -o option to change internal variables
runtests.pl has lots of internal variables one might want to
change in certain situations, but adding a dedicated option
for every single one of them isn't practical.

Usage:
./runtests.pl -o TESTDIR=$privoxy_curl_test_dir -o HOSTIP=10.0.0.1 ...
2021-02-09 00:28:29 +01:00
Fabian Keil
e6d55f647d
runtests.pl: cleanups
- show the summarized test result in the last line of the report
- do not use $_ after mapping it to a named variable
  Doing that makes the code harder to follow.
- log the restraints sorted by the number of their occurrences
- fix language when logging restraints that only occured once
- let runhttpserver() use $TESTDIR instead of $srcdir
  ... so it works if a non-default $TESTDIR is being used.
2021-02-09 00:28:29 +01:00
Fabian Keil
3f0bef2b53
runtests.pl: add an -E option to specify an exclude file
It can contain additional restraints for test numbers,
keywords and tools.

The idea is to let third parties like the Privoxy project
distribute an exclude file with their tarballs that specifies
which curl tests are not expected to work when using Privoxy
as a proxy, without having to fork the whole curl test suite.

The syntax could be changed to be extendable and maybe
more closely reflect the "curl test" syntax. Currently
it's a bunch of lines like these:

test:$TESTNUMBER:Reason why this test with number $TESTNUMBER should be skipped
keyword:$KEYWORD:Reason why tests whose keywords contain the $KEYWORD should be skipped
tool:$TOOL:Reason why tests with tools that contain $TOOL should be skipped

To specify multiple $TESTNUMBERs, $KEYWORDs and $TOOLs
on a single line, split them with commas.
2021-02-09 00:28:28 +01:00
Fabian Keil
b47ee58fb7
runtests.pl: add -L parameter to require additional perl libraries
This is useful to change the behaviour of the script without
having to modify the file itself, for example to use a custom
compareparts() function that ignores header differences that
are expected to occur when an external proxy is being used.

Such differences are proxy-specific and thus the modifications
should be maintained together with the proxy.
2021-02-09 00:28:28 +01:00
Fabian Keil
4421359e77
runtests.pl: add a -P option to specify an external proxy
... that should be used when executing the tests.

The assumption is that the proxy is an HTTP proxy.

This option should be used together with -L to provide
a customized compareparts() version that knows which
proxy-specific header differences should be ignored.

This option doesn't work for all test types yet.
2021-02-09 00:28:28 +01:00
Fabian Keil
124198631b
tests: fixup several tests
missing CRs and modified %hostip

lib556/test556: use a real HTTP version to make test reuse more convenient

make sure the weekday in Date headers matches the date

test61: replace stray "^M" (5e 4d) at the end of a cookie with a '^M' (0d)

Gets the test working with external proxies like Privoxy again.

Closes #6463
2021-02-09 00:28:19 +01:00
Daniel Stenberg
e32601e13a
ftp: avoid SIZE when asking for a TYPE A file
... as we ignore it anyway because servers don't report the correct size
and proftpd even blatantly returns a 550.

Updates a set of tests accordingly.

Reported-by: awesomenode on github
Fixes #6564
Closes #6565
2021-02-04 23:06:03 +01:00
Daniel Stenberg
2605a6fabb
copyright: fix year ranges in need of updates 2021-02-03 08:02:36 +01:00
Dmitry Wagin
796ce293de
http: improve AWS HTTP v4 Signature auth
- Add support services without region and service prefixes in
the URL endpoint (ex. Min.IO, GCP, Yandex Cloud, Mail.Ru Cloud Solutions, etc)
by providing region and service parameters via aws-sigv4 option.
- Add [:region[:service]] suffix to aws-sigv4 option;
- Fix memory allocation errors.
- Refactor memory management.
- Use Curl_http_method instead() STRING_CUSTOMREQUEST.
- Refactor canonical headers generating.
- Remove repeated sha256_to_hex() usage.
- Add some docs fixes.
- Add some codestyle fixes.
- Add overloaded strndup() for debug - curl_dbg_strndup().
- Update tests.

Closes #6524
2021-01-30 23:48:22 +01:00
Daniel Stenberg
f6d13090d4
copyright: fix missing year (range) updates 2021-01-29 14:35:13 +01:00
Daniel Stenberg
553f286a8f
tool_writeout: make %{errormsg} blank for no errors
Closes #6539
2021-01-28 07:55:25 +01:00
Jay Satiro
782143ee81 test410: fix for windows
- Pass the very long request header via file instead of command line.

Prior to this change the 49k very long request header string was passed
via command line and on Windows that is too long so it was truncated and
the test would fail (specifically msys CI).

Closes https://github.com/curl/curl/pull/6516
2021-01-25 23:28:02 -05:00
Daniel Stenberg
8335c6417e
hostip: remove conn->data from resolver functions
This also moves the 'async' struct from the connectdata struct into the
Curl_easy struct, which seems like a better home for it.

Closes #6497
2021-01-22 08:25:09 +01:00
Fabian Keil
0015e06151
http: get CURLOPT_REQUEST_TARGET working with a HTTP proxy
Added test 1613 to verify.

Closes #6490
2021-01-20 15:21:13 +01:00
Fabian Keil
6fcf22654e
Add test1181: Proxy request with --proxy-header "Connection: Keep-Alive" 2021-01-19 11:41:23 +01:00
Fabian Keil
3bccc182a2
Add test1180: Proxy request with -H "Proxy-Connection: Keep-Alive"
At the moment the test fails as curl sends two Proxy-Connection
headers.
2021-01-19 11:41:23 +01:00
Daniel Stenberg
f451206266
runtests: preprocess DISABLED to allow conditionals
... with this function provided, we can disable tests for specific
environments and setups directly within this file.

Closes #6477
2021-01-19 10:04:41 +01:00
Daniel Stenberg
bbfad7e8a1
runtests: turn preprocessing into a separate function
... and remove all other variable substitutions as they're now done once
and for all in the preprocessor.
2021-01-19 10:04:41 +01:00
Daniel Stenberg
11b2dd86d3
test410: verify HTTPS GET with a 49K request header
skip test 410 for mesalink in the CI as it otherwise hangs "forever"
2021-01-18 16:10:34 +01:00
Daniel Stenberg
215db086e0
lib: pass in 'struct Curl_easy *' to most functions
... in most cases instead of 'struct connectdata *' but in some cases in
addition to.

- We mostly operate on transfers and not connections.

- We need the transfer handle to log, store data and more. Everything in
  libcurl is driven by a transfer (the CURL * in the public API).

- This work clarifies and separates the transfers from the connections
  better.

- We should avoid "conn->data". Since individual connections can be used
  by many transfers when multiplexing, making sure that conn->data
  points to the current and correct transfer at all times is difficult
  and has been notoriously error-prone over the years. The goal is to
  ultimately remove the conn->data pointer for this reason.

Closes #6425
2021-01-17 23:56:09 +01:00
Daniel Stenberg
30e7641d7d
test1918: verify curl_easy_option_by_name() and curl_easy_option_by_id()
... and as a practical side-effect, make sure that the
Curl_easyopts_check() function is asserted in debug builds, which we
want to detect mismatches between the options list in easyoptions.c and
the options in curl.h

Found-by: Gisle Vanem
Bug: 08e8455ddd (commitcomment-45991815)

Closes #6461
2021-01-15 17:03:29 +01:00
Daniel Stenberg
1123a0eb24
mime: make sure setting MIMEPOST to NULL resets properly
... so that a function can first use MIMEPOST and then set it to NULL to
reset it back to a blank POST.

Added test 584 to verify the fix.

Reported-by: Christoph M. Becker

Fixes #6455
Closes #6456
2021-01-15 08:33:27 +01:00
Fabian Keil
4f61fd87b2
misc: fix typos
Bug: https://curl.se/mail/lib-2021-01/0063.html
Closes #6434
2021-01-11 15:12:25 +01:00
Daniel Stenberg
060658f634
test179: use consistent header line endings
... to make "Hyper mode" work better.
2021-01-08 10:57:48 +01:00
Daniel Stenberg
ab525c059e
http: have CURLOPT_FAILONERROR fail after all headers
... so that Retry-After and other meta-content can still be used.

Added 1634 to verify. Adjusted test 194 and 281 since --fail now also
includes the header-terminating CRLF in the output before it exits.

Fixes #6408
Closes #6409
2021-01-06 15:14:40 +01:00
Daniel Stenberg
9d7281ff17
global_init: debug builds allocates a byte in init
... to make build tools/valgrind warn if no curl_global_cleanup is
called.

This is conditionally only done for debug builds with the env variable
CURL_GLOBAL_INIT set.

Closes #6410
2021-01-06 15:13:45 +01:00
Daniel Stenberg
75f7ab28d2
lib/unit tests: add missing curl_global_cleanup() calls 2021-01-06 15:13:45 +01:00
Daniel Stenberg
80c5a838ce
test66: disable with Hyper
...as Hyper doesn't support HTTP/0.9
2021-01-05 17:27:05 +01:00
Daniel Stenberg
82e6d34803
test1633: set appropriate name
"--retry with a 429 response and Retry-After:"
2021-01-04 17:58:05 +01:00
Daniel Stenberg
648712eec1
httpauth: make multi-request auth work with custom port
When doing HTTP authentication and a port number set with CURLOPT_PORT,
the code would previously have the URL's port number override as if it
had been a redirect to an absolute URL.

Added test 1568 to verify.

Reported-by: UrsusArctos on github
Fixes #6397
Closes #6400
2021-01-02 23:40:47 +01:00
Emil Engler
725ec470e2
language: s/behaviour/behavior/g
We currently use both spellings the british "behaviour" and the american
"behavior". However "behavior" is more used in the project so I think
it's worth dropping the british name.

Closes #6395
2021-01-02 23:35:59 +01:00
Olaf Hering
f2ed79d8ee
docs/examples: adjust prototypes for CURLOPT_READFUNCTION
The type of the buffer in curl_read_callback is 'char *', not 'void *'.

Signed-off-by: Olaf Hering <olaf@aepfle.de>
Closes #6392
2020-12-30 22:20:24 +01:00
Daniel Stenberg
b9746575a9
tests/unit: fix empty statements with no effect
... by making macros use "do {} while(0)"
2020-12-29 23:02:43 +01:00
Paul Groke
8324dc8b1a
dns: extend CURLOPT_RESOLVE syntax for adding non-permanent entries
Extend the syntax of CURLOPT_RESOLVE strings: allow using a '+' prefix
(similar to the existing '-' prefix for removing entries) to add
DNS cache entries that will time out just like entries that are added
by libcurl itself.

Append " (non-permanent)" to info log message in case a non-permanent
entry is added.

Adjust relevant comments to reflect the new behavior.

Adjust documentation.

Extend unit1607 to test the new functionality.

Closes #6294
2020-12-29 17:49:36 +01:00
XhmikosR
aadc75626b
misc: assorted typo fixes
Closes #6375
2020-12-26 23:54:25 +01:00
Daniel Stenberg
e99e5ab820
tests/badsymbols.pl: ignore stand-alone single hash lines
Bug: https://curl.se/mail/lib-2020-12/0084.html
Reported-by: Dennis Clarke
Assisted-by: Jay Satiro

Closes #6355
2020-12-22 23:12:10 +01:00
Daniel Stenberg
a93c647de8
examples/libtest: add .checksrc to dist
... so that (auto)builds from tarballs also get the correct instructions.

Fixes #6176
Closes #6353
2020-12-21 17:06:06 +01:00
Daniel Stenberg
58c4b8d636
test: verify new --write-out variables
Extended test 1029 and added 1188
2020-12-21 16:38:53 +01:00
Daniel Stenberg
f7a65979f3
test970: adapted to the new internal order of variables 2020-12-21 16:38:52 +01:00
Matthias Gatto
ebdb5f23cc
tests: add very simple AWS HTTP v4 Signature test
Signed-off-by: Matthias Gatto <matthias.gatto@outscale.com>
2020-12-21 16:28:03 +01:00
Daniel Stenberg
a7696c7343
curl: add --create-file-mode [mode]
This option sets the (octal) mode to use for the remote file when one is
created, using the SFTP, SCP or FILE protocols. When not set, the
default is 0644.

Closes #6244
2020-12-21 10:52:41 +01:00
Jay Satiro
7f170a0f5f badsymbols.pl: Add verbose mode -v
Use -v as the first option to enable verbose mode which will show source
input, extracted symbol and line info. For example:

Source: ./../include/curl/typecheck-gcc.h
Symbol: curlcheck_socket_info(info)
Line #423: #define curlcheck_socket_info(info)                     \

Ref: https://curl.se/mail/lib-2020-12/0084.html

Closes https://github.com/curl/curl/pull/6349
2020-12-20 15:59:18 -05:00
Daniel Stenberg
debf23eead
mqtt: handle POST/PUBLISH without a set POSTFIELDSIZE
Detected by OSS-Fuzz
Bug: https://bugs.chromium.org/p/oss-fuzz/issues/detail?id=28735

Added test 1916 and 1917 to verify.

Closes #6338
2020-12-18 12:54:05 +01:00
Daniel Stenberg
c7dd08ce58
tests: updated tests for Hyper 2020-12-18 09:58:04 +01:00
Daniel Stenberg
ff4d2c2a05
test1522: add debug tracing
I used this to track down some issues and I figured I could just as well
keep this extra logging in here for future needs.

Closes #6331
2020-12-17 16:55:56 +01:00
Daniel Stenberg
a5bc272223
http: show the request as headers even when split-sending
When the initial request isn't possible to send in its entirety, the
remainder of request would be delivered to the debug callback as data
and would wrongly be counted internally as body-bytes sent.

Extended test 1295 to verify.

Closes #6328
2020-12-17 14:00:27 +01:00
Daniel Stenberg
3abfadfc19
test1198/9: add two mqtt publish tests without payload lengths
Closes #6335
2020-12-17 13:55:02 +01:00
Daniel Stenberg
30fc1cc7db
tests/mqttd: extract the client id from the correct offset
Closes #6334
2020-12-17 10:15:31 +01:00
Daniel Stenberg
354bc95d96
test1272: fix line ending
Follow-up to f24784f914
2020-12-15 17:38:42 +01:00
Daniel Stenberg
f24784f914
test1272: test gophers 2020-12-15 12:58:19 +01:00
Daniel Stenberg
48b85c46f1
runtests: add support for gophers, gopher over TLS 2020-12-15 12:58:19 +01:00
Daniel Stenberg
eddae97406
test1564/1565: require the 'wakeup' feature to run
Fixes #6299
Fixes #6300
Closes #6301
2020-12-11 17:42:18 +01:00
Daniel Stenberg
34bb59a868
runtests: add 'wakeup' as a feature 2020-12-11 17:42:17 +01:00
Daniel Stenberg
769a4687df
tests/server/disabled: add "wakeup"
To allow the test suite to know if wakeup support is disabled in the
build.
2020-12-11 17:42:17 +01:00
Daniel Stenberg
6cd066f64b
lib1564/5: verify that curl_multi_wakeup returns OK 2020-12-11 17:42:17 +01:00
Daniel Stenberg
3a63c190bb
tests: make --libcurl tests only test FTP options if ftp enabled
Adjust six --libcurl tests to only check the FTP option if FTP is
actually present in the build.

Fixes #6303
Closes #6305
2020-12-11 17:28:35 +01:00
Daniel Stenberg
3e92799171
runtests.pl: fix "uninitialized value" warning
follow-up to e12825c642
2020-12-11 13:53:23 +01:00
Daniel Stenberg
e12825c642
runtests: add support for %if [feature] conditions
... to make tests run differently or expect different results depending
on what features that are present or not in curl.

Bonus: initial minor 'Hyper' awareness but nothing is using that yet

Closes #6304
2020-12-11 13:16:02 +01:00
Jakub Zakrzewski
cfea4f2f4d
cmake: don't use reserved target name 'test'
CMake up to 3.10 always reserves this name

Fixes #6257
Closes #6258
2020-12-07 10:33:35 +01:00
Daniel Stenberg
ec9cc725d5
ftp: CURLOPT_FTP_SKIP_PASV_IP by default
The command line tool also independently sets --ftp-skip-pasv-ip by
default.

Ten test cases updated to adapt the modified --libcurl output.

Bug: https://curl.se/docs/CVE-2020-8284.html
CVE-2020-8284

Reported-by: Varnavas Papaioannou
2020-12-07 08:38:05 +01:00
Daniel Stenberg
abd846c374
urlapi: don't accept blank port number field without scheme
... as it makes the URL parser accept "very-long-hostname://" as a valid
host name and we don't want that. The parser now only accepts a blank
(no digits) after the colon if the URL starts with a scheme.

Reported-by: d4d on hackerone

Closes #6283
2020-12-07 00:50:49 +01:00
Marc Hoersken
753a2c758a
tests/util.py: fix compatibility with Python 2
Backporting the Python 3 implementation of setStream
to ClosingFileHandler as a fallback within Python 2.

Reported-by: Jay Satiro

Fixes #6259
Closes #6270
2020-12-03 20:57:39 +01:00
Daniel Stenberg
b6b535994e
splay: rename Curl_splayremovebyaddr to Curl_splayremove
... and remove the old unused proto for the old Curl_splayremove
version.

Closes #6269
2020-12-01 08:09:51 +01:00
Marc Hoersken
227daceabe
tests/server/tftpd.c: close upload file in case of abort
Commit c353207 removed the closing right after do_tftp
which covered the case of abort. This handles that case.

Reviewed-by: Jay Satiro
Reviewed-by: Daniel Stenberg

Follow up to #6209
Closes #6234
2020-11-28 19:19:18 +01:00
Daniel Stenberg
72ae6737e0 test506: make it not run in c-ares builds
As the asynch nature of it may trigger events in another order. A c-ares
upgrade made it break.

Reported-by: Marc Hörsken
Fixes #6247
2020-11-26 17:24:55 +01:00
Daniel Stenberg
082422b189 runtests: make 'c-ares' a "feature" to depend on
... also added to the docs.
2020-11-26 17:24:24 +01:00
Rikard Falkeborn
920f49a20b
infof/failf calls: fix format specifiers
Update a few format specifiers to match what is being printed.

Closes #6241
2020-11-24 13:18:41 +01:00