Commit Graph

407 Commits

Author SHA1 Message Date
Fabian Keil
e1e9d5af73
test374: gif data without new line at the end
Closes #8239
2022-01-10 15:54:54 +01:00
Daniel Stenberg
d1237ac906
url: given a user in the URL, find pwd for that user in netrc
Add test 380 and 381 to verify, edited test 133

Reported-by: Manfred Schwarb
Fixes #8241
Closes #8243
2022-01-09 14:16:41 +01:00
Daniel Stenberg
764e4f066d
tool_findfile: check ~/.config/curlrc too
... after the initial checks for .curlrc and if XDG_CONFIG_HOME is not
set, use $HOME and $CURL_HOME to check if ~/.config/curlrc is present.

Add test 436 to verify

Reported-by: Sandro Jaeckel
Fixes #8208
Closes #8213
2022-01-07 10:10:49 +01:00
Fabian Keil
5da22337fe
Add test373: multiple chunks with binary zeros 2022-01-07 10:01:44 +01:00
Fabian Keil
33d0503103
Add test372: binary zero in data element 2022-01-07 10:01:44 +01:00
Daniel Stenberg
dbbf3609f3
test719-721: verify SOCKS details
Using the new verify/socks details
2022-01-03 22:32:05 +01:00
Daniel Stenberg
effd2bd7ba
socks5: use appropriate ATYP for numerical IP address host names
When not resolving the address locallly (known as socks5h).

Add test 719 and 720 to verify.

Reported-by: Peter Piekarski
Fixes #8216
Closes #8217
2022-01-03 22:32:05 +01:00
Daniel Stenberg
ee2ca5826b
tool_operate: warn if too many output arguments were found
More output instructions than URLs is likely a user error.

Add test case 371 to verify

Closes #8210
2022-01-03 15:55:38 +01:00
Vladimir Panteleev
2c1dbc1af0
tests: Add test for CURLOPT_HTTP200ALIASES 2021-12-21 08:45:06 +01:00
Daniel Stenberg
2b3dd01b77
multi: handle errors returned from socket/timer callbacks
The callbacks were partially documented to support this. Now the
behavior is documented and returning error from either of these
callbacks will effectively kill all currently ongoing transfers.

Added test 530 to verify

Reported-by: Marcelo Juchem
Fixes #8083
Closes #8089
2021-12-06 14:55:52 +01:00
Patrick Monnerat
b20b364764
mime: use percent-escaping for multipart form field and file names
Until now, form field and file names where escaped using the
backslash-escaping algorithm defined for multipart mails. This commit
replaces this with the percent-escaping method for URLs.

As this may introduce incompatibilities with server-side applications, a
new libcurl option CURLOPT_MIME_OPTIONS with bitmask
CURLMIMEOPT_FORMESCAPE is introduced to revert to legacy use of
backslash-escaping. This is controlled by new cli tool option
--form-escape.

New tests and documentation are provided for this feature.

Reported by: Ryan Sleevi
Fixes #7789
Closes #7805
2021-11-15 10:40:03 +01:00
Daniel Stenberg
f0b7099a10
multi: shut down CONNECT in Curl_detach_connnection
... to prevent a lingering pointer that would lead to a double-free.

Added test 1939 to verify.

Reported-by: Stephen M. Coakley
Fixes #7982
Closes #7986
2021-11-11 08:42:31 +01:00
Marc Hoersken
b3e1ed3160
tests: add Schannel-specific tests and disable unsupported ones
Adds Schannel variants of SSLpinning tests that include the option
--ssl-revoke-best-effort to ignore certificate revocation check
failures which is required due to our custom test CA certificate.

Disable the original variants if the Schannel backend is enabled.

Also skip all IDN tests which are broken while using an msys shell.

This is a step to simplify test exclusions for Windows and MinGW.

Reviewed-by: Jay Satiro
Reviewed-by: Marcel Raad
Reviewed-by: Daniel Stenberg
Closes #7968
2021-11-09 06:34:33 +01:00
Daniel Stenberg
cc71d35265
tool_operate: a failed etag save now only fails that transfer
When failing to create the output file for saving an etag, only fail
that particular single transfer and allow others to follow.

In a serial transfer setup, if no transfer at all is done due to them
all being skipped because of this error, curl will output an error
message and return exit code 26.

Added test 369 and 370 to verify.

Reported-by: Earnestly on github
Ref: #7942
Closes #7945
2021-11-04 13:56:15 +01:00
Daniel Stenberg
c75f14cb9d
test1185: verify checksrc
Closes #7866
2021-10-18 23:52:30 +02:00
Abhinav Singh
8c6f126279
aws-sigv4: make signature work when post data is binary
User sets the post fields size for binary data.  Hence, we should not be
using strlen on it.

Added test 1937 and 1938 to verify.

Closes #7844
2021-10-15 08:46:23 +02:00
Daniel Stenberg
e38d2e5a50
test368: verify dash is appended for "-r [num]"
Follow-up to 8758a26f88
2021-10-11 22:52:30 +02:00
Daniel Stenberg
87907d56f0
http: fix Basic auth with empty name field in URL
Add test 367 to verify.

Reported-by: Rick Lane
Fixes #7819
Closes #7820
2021-10-06 22:45:52 +02:00
Jeffrey Tolar
5f563495f1
CURLOPT_MAXLIFETIME_CONN: maximum allowed lifetime for conn reuse
... and close connections that are too old instead of reusing them.

By default, this behavior is disabled.

Bug: https://curl.se/mail/lib-2021-09/0058.html
Closes #7751
2021-10-06 14:38:59 +02:00
Max Dymond
a517378de5
CURLOPT_PREREQFUNCTION: add new callback
Triggered before a request is made but after a connection is set up

Changes:

- callback: Update docs and callback for pre-request callback
- Add documentation for CURLOPT_PREREQDATA and CURLOPT_PREREQFUNCTION,
- Add redirect test and callback failure test
- Note that the function may be called multiple times on a redirection
- Disable new 2086 test due to Windows weirdness

Closes #7477
2021-09-27 17:16:43 +02:00
Mats Lindestam
d1e7d9197b
libssh2: add SHA256 fingerprint support
Added support for SHA256 fingerprint in command line curl and in
libcurl.

Closes #7646
2021-09-26 23:20:53 +02:00
Daniel Stenberg
883cf374d8
test897: verify delivery of IMAP post-body header content
The "content" is delivered as "body" by curl, but the envelope continues
after the body and the rest of it should be delivered as header.

The IMAP server can now get 'POSTFETCH' set to include more data to
include after the body and test 897 is done to verify that such "extra"
header data is in fact delivered by curl as header.

Ref: #7284 but fails to reproduce the issue

Closes #7748
2021-09-21 08:18:42 +02:00
Daniel Stenberg
beb8990d93
http: fix the broken >3 digit response code detection
When the "reason phrase" in the HTTP status line starts with a digit,
that was treated as the forth response code digit and curl would claim
the response to be non-compliant.

Added test 1466 to verify this case.

Regression brought by 5dc594e44f
Reported-by: Glenn de boer
Fixes #7738
Closes #7739
2021-09-17 17:52:00 +02:00
Patrick Monnerat
8ef147c436
ftp,imap,pop3,smtp: reject STARTTLS server response pipelining
If a server pipelines future responses within the STARTTLS response, the
former are preserved in the pingpong cache across TLS negotiation and
used as responses to the encrypted commands.

This fix detects pipelined STARTTLS responses and rejects them with an
error.

CVE-2021-22947

Bug: https://curl.se/docs/CVE-2021-22947.html
2021-09-13 16:51:31 +02:00
Patrick Monnerat
364f174724
ftp,imap,pop3: do not ignore --ssl-reqd
In imap and pop3, check if TLS is required even when capabilities
request has failed.

In ftp, ignore preauthentication (230 status of server greeting) if TLS
is required.

Bug: https://curl.se/docs/CVE-2021-22946.html

CVE-2021-22946
2021-09-13 16:51:31 +02:00
Daniel Stenberg
4a8fa48946
http_proxy: fix the User-Agent inclusion in CONNECT
It should not refer to the uagent string that is allocated and created
for the end server http request, as that pointer may be cleared on
subsequent CONNECT requests.

Added test case 1184 to verify.

Reported-by: T200proX7 on github
Fixes #7705
Closes #7707
2021-09-13 12:29:59 +02:00
Daniel Stenberg
eb2a5171da
curl: stop retry if Retry-After: is longer than allowed
If Retry-After: specifies a period that is longer than what fits within
--retry-max-time, then stop retrying immediately.

Added test 366 to verify.

Reported-by: Kari Pahula
Fixes #7675
Closes #7676
2021-09-06 14:46:45 +02:00
Daniel Stenberg
424051d78b
connect: get local port + ip also when reusing connections
Regression. In d6a37c23a3 (7.75.0) we removed the duplicated storage
(connection + easy handle), so this info needs be extracted again even
for re-used connections.

Add test 435 to verify

Reported-by: Max Dymond
Fixes #7660
Closes #7662
2021-09-02 23:42:18 +02:00
Daniel Stenberg
e8c8775eaa
test365: verify response with chunked AND Content-Length headers 2021-08-31 10:04:03 +02:00
Daniel Stenberg
c495dcd02e
http: consider cookies over localhost to be secure
Updated test31.
Added test 392 to verify secure cookies used for http://localhost

Reviewed-by: Daniel Gustafsson
Fixes #6733
Closes #7263
2021-08-10 11:20:21 +02:00
Daniel Stenberg
a5ab72d5ed
test677: IMAP CONNECT_ONLY, custom command and then exit
Adjusted ftpserver.pl to add support for the IMAP IDLE command

Adjusted test 660 to sync with the fix
2021-06-24 09:07:40 +02:00
Daniel Stenberg
bbbc5de93f
netrc: skip 'macdef' definitions
Add test 494 to verify

Reported-by: Harry Sintonen
Fixes #7238
Closes #7244
2021-06-13 23:44:41 +02:00
Gealber Morales
791937b881
mqtt: add support for username and password
Minor-edits-by: Daniel Stenberg
Added test 2200 to 2205

Closes #7243
2021-06-12 23:50:13 +02:00
Daniel Stenberg
3ac9b80525
test644: remove as duplicate of test 587
Closes #7208
2021-06-08 16:07:56 +02:00
Daniel Stenberg
265b14d6b3
metalink: remove
Warning: this will make existing curl command lines that use metalink to
stop working.

Reasons for removal:

1. We've found several security problems and issues involving the
   metalink support in curl. The issues are not detailed here. When
   working on those, it become apparent to the team that several of the
   problems are due to the system design, metalink library API and what
   the metalink RFC says. They are very hard to fix on the curl side
   only.

2. The metalink usage with curl was only very briefly documented and was
   not following the "normal" curl usage pattern in several ways, making
   it surprising and non-intuitive which could lead to further security
   issues.

3. The metalink library was last updated 6 years ago and wasn't so
   active the years before that either. An unmaintained library means
   there's a security problem waiting to happen. This is probably reason
   enough.

4. Metalink requires an XML parsing library, which is complex code (even
   the smaller alternatives) and to this day often gets security
   updates.

5. Metalink is not a widely used curl feature. In the 2020 curl user
   survey, only 1.4% of the responders said that they'd are using it. In
   2021 that number was 1.2%. Searching the web also show very few
   traces of it being used, even with other tools.

6. The torrent format and associated technology clearly won for
   downloading large files from multiple sources in parallel.

Cloes #7176
2021-06-07 08:14:25 +02:00
Daniel Stenberg
510e6e9a19
tests/data/Makefile.inc: error: trailing backslash on last line
Follow-up to d8dcb399b8
2021-06-04 16:52:34 +02:00
Alexis Vachette
d8dcb399b8
url: bad CURLOPT_CONNECT_TO syntax now returns error
Added test 3020 to verify

Closes #7183
2021-06-04 14:08:30 +02:00
Alexis Vachette
cb39b4b7c4
hostip: bad CURLOPT_RESOLVE syntax now returns error
Added test 3019
Fixes #7170
Closes #7174
2021-06-03 13:53:18 +02:00
Harry Sintonen
8ccf75532b
mqtt: detect illegal and too large file size
Add test 3017 and 3018 to verify.
Closes #7166
2021-06-02 13:34:17 +02:00
Jacob Hoffman-Andrews
a62e6435f4
rustls: switch read_tls and write_tls to callbacks
And update to 0.6.0, including a rename from session to connection for
many fields.

Closes #7071
2021-05-24 16:40:59 +02:00
Daniel Stenberg
51c0ebcff2
http: deal with partial CONNECT sends
Also added 'CURL_SMALLSENDS' to make Curl_write() send short packets,
which helped verifying this even more.

Add test 363 to verify.

Reported-by: ustcqidi on github
Fixes #6950
Closes #7024
2021-05-08 10:49:16 +02:00
Gilles Vollant
77fc3859b2 SSL: support in-memory CA certs for some backends
- New options CURLOPT_CAINFO_BLOB and CURLOPT_PROXY_CAINFO_BLOB to
  specify in-memory PEM certificates for OpenSSL, Schannel (Windows)
  and Secure Transport (Apple) SSL backends.

Prior to this change PEM certificates could only be imported from a file
and not from memory.

Co-authored-by: moparisthebest@users.noreply.github.com

Ref: https://github.com/curl/curl/pull/4679
Ref: https://github.com/curl/curl/pull/5677
Ref: https://github.com/curl/curl/pull/6109

Closes https://github.com/curl/curl/pull/6662
2021-05-05 02:29:16 -04:00
Patrick Monnerat
34cf40321c
bufref: buffer reference support
A struct bufref holds a buffer pointer, a data size and a destructor.
When freed or its contents are changed, the previous buffer is implicitly
released by the associated destructor. The data size, although not used
internally, allows binary data support.

A unit test checks its handling methods: test 1661

Closes #6654
2021-04-22 09:05:53 +02:00
Daniel Stenberg
6fc805d0c1
test972: verify the json output with jsonlint
Make sure one of the azure jobs has jsonlint installed so that the test
runs there.

Ref: #6905
2021-04-17 11:03:40 +02:00
Daniel Stenberg
520bd5225c
cookie: CURLOPT_COOKIEFILE set to NULL switches off cookies
Add test 676 to verify that setting CURLOPT_COOKIEFILE to NULL again clears
the cookiejar from memory.

Reported-by: Stefan Karpinski
Fixes #6889
Closes #6891
2021-04-14 23:09:36 +02:00
Daniel Stenberg
605aa03ac1
http_proxy: only loop on 407 + close if we have credentials
... to fix the retry-loop.

Add test 718 to verify.

Reported-by: Daniel Kurečka
Fixes #6828
Closes #6850
2021-04-05 22:49:07 +02:00
Luke Granger-Brown
6d930d7306
file: support GETing directories again
After 957bc1881e, we no longer compute an
expected_size for directories. This has the upshot that when we compare
even an empty Range with the available size, we fail.

This brings back the previous behaviour, which was to succeed, but with
empty content. This also removes the "Accept-ranges: bytes" header,
which is nonsensical on directories.

Adds test 3016
Fixes #6845
Closes #6846
2021-04-05 12:22:07 +02:00
Viktor Szakats
7214288898
transfer: strip credentials from the auto-referer header field
Added test 2081 to verify.

CVE-2021-22876

Bug: https://curl.se/docs/CVE-2021-22876.html
2021-03-28 23:19:55 +02:00
Daniel Stenberg
6d176bee59
http: make 416 not fail with resume + CURLOPT_FAILONERRROR
When asked to resume a download, libcurl will convert that to HTTP logic
and if then the entire file is already transferred it will result in a
416 response from the HTTP server. With CURLOPT_FAILONERRROR set in that
scenario, it should *not* lead to an error return.

Updated test 1156, added test 1273

Reported-by: Jonathan Watt
Fixes #6740
Closes #6753
2021-03-17 08:26:46 +01:00
Daniel Stenberg
7b2f0676c2
FTP: allow SIZE to fail when doing (resumed) upload
Added test 362 to verify.

Reported-by: Jordan Brown
Regression since 7ea2e1d0c5 (7.73.0)
Fixes #6715
Closes #6725
2021-03-11 16:13:11 +01:00