13674 Commits

Author SHA1 Message Date
Stefan Eissing
59e2c78af3 http_chunks: fix the accounting of consumed bytes
Prior to this change chunks were handled correctly although in verbose
mode libcurl could incorrectly warn of "Leftovers after chunking" even
if there were none.

Reported-by: Michael Kaufmann

Fixes https://github.com/curl/curl/issues/12937
Closes https://github.com/curl/curl/pull/12939
2024-02-18 02:16:29 -05:00
Stefan Eissing
0e2ffa3632 file: use xfer buf for file:// transfers
- For file:// transfers use the multi handle's transfer buffer for
  up- and downloads.

Prior to this change a6c9a33 (precedes 8.6.0) changed the file://
transfers to use a smaller stack based buffer, and that caused a
significant performance decrease in Windows.

Bug: https://github.com/curl/curl/issues/12750#issuecomment-1920103086
Reported-by: edmcln@users.noreply.github.com

Closes https://github.com/curl/curl/pull/12932
2024-02-18 02:12:45 -05:00
Stefan Eissing
e87751d69a vtls: fix tls proxy peer verification
- When verifying a proxy certificate for an ip address, use the correct
  ip family.

Prior to this change the "connection" ip family was used, which was not
necessarily the same.

Reported-by: HsiehYuho@users.noreply.github.com

Fixes https://github.com/curl/curl/issues/12831
Closes https://github.com/curl/curl/pull/12931
2024-02-16 18:00:21 -05:00
Jay Satiro
5691a6cf36 transfer: improve Windows SO_SNDBUF update limit
- Change the 1 second SO_SNDBUF update limit from per transfer to per
  connection.

Prior to this change many transfers over the same connection could cause
many SO_SNDBUF updates made to that connection per second, which was
unnecessary.

Closes https://github.com/curl/curl/pull/12911
2024-02-13 03:46:17 -05:00
Jay Satiro
24d6c2889f schannel: fix hang on unexpected server close
- Treat TLS connection close (either due to a close_notify from the
  server or just closed due to receiving 0) as pending data.

This is because in some cases schannel_recv knows the connection is
closed but has to return actual pending data so it can't return 0 or an
error to indicate no more data. In this case schannel_recv must be
called again, which only happens if readwrite_data sees that there is
still pending data.

Prior to this change if the total size of the body that libcurl expected
to receive from the server was unknown then it was possible under some
network conditions that libcurl would hang waiting to receive more data,
when in fact a close_notify alert indicating no more data would be sent
was already processed.

Fixes https://github.com/curl/curl/issues/12894
Closes https://github.com/curl/curl/pull/12910
2024-02-13 03:45:21 -05:00
MAntoniak
f8bd04ec50
mbedtls: use mbedtls_ssl_conf_{min|max}_tls_version
... instead of the deprecated mbedtls_ssl_conf_{min|max}_version

Closes #12905
2024-02-10 14:21:50 +01:00
MAntoniak
07e5b3ea60
mbedtls: fix building when MBEDTLS_X509_REMOVE_INFO flag is defined
Closes #12904
2024-02-09 16:00:43 +01:00
Stefan Eissing
d8f01e0985
ftp: fix socket wait activity in ftp_domore_getsock
- when waiting on the data connection, always add the control socket to
  the pollset on state STOP or let the pingpong add the socket according
  to its needs.

Reported-by: Fabian Vogt
Fixes #12901
Closes #12913
2024-02-09 15:57:06 +01:00
Stefan Eissing
476adfeac0
multi: add xfer_buf to multi handle
- can be borrowed by transfer during recv-write operation
- needs to be released before borrowing again
- adjustis size to `data->set.buffer_size`
- used in transfer.c readwrite_data()

Closes #12805
2024-02-09 09:43:50 +01:00
Stefan Eissing
e7fd32b9ac
ftp: do lineend conversions in client writer
- remove the ftp special handling from sendf.c
- let ftp_do() add a client writer that does
  the linened conversions
- change the lineend conversion to no longer
  modify the passed buffer, but write smaller
  chunks to the next cwriter instead. The
  inefficiency of this will be mitigated once
  we add output buffering for all client writes.

Closes #12878
2024-02-08 23:20:56 +01:00
Stefan Eissing
6984aa3a45
ftp: tracing improvements
- trace socketindex for connection filters when not the first
- trace socket fd in tcp
- trace pollset adjusts in vtls

Closes #12902
2024-02-08 23:14:33 +01:00
Stefan Eissing
ed09a99af5
vtls: revert "receive max buffer" + add test case
- add test_05_04 for requests using http/1.0, http/1.1 and h2 against an
  Apache resource that does an unclean TLS shutdown.
- revert special workarund in openssl.c for suppressing shutdown errors
  on multiplexed connections
- vlts.c restore to its state before 9a90c9dd64d2f03601833a70786d485851bd1b53

Fixes #12885
Fixes #12844

Closes #12848
2024-02-07 14:46:41 +01:00
Daniel Stenberg
142ac257b3
lib: convert Curl_get_line to use dynbuf
Create the line in a dynbuf. Aborts the reading of the file on
errors. Avoids having to always allocate maximum amount from the
start. Avoids direct malloc.

Closes #12846
2024-02-07 09:22:10 +01:00
Stefan Eissing
c177e1944c
https-proxy: use IP address and cert with ip in alt names
- improve info logging when peer verification fails to indicate
  if DNS name or ip address has been tried to match
- add test case for contacting https proxy with ip address
- add pytest env check on loaded credentials and re-issue
  when they are no longer valid
- disable proxy ip address test for bearssl, since not supported there

Ref: #12831
Closes #12838
2024-02-06 10:10:14 +01:00
Peter Krefting
174c89ec3f
version: allow building with ancient libpsl
The psl_check_version_number() API was added in libpsl 0.11.0. CentOS 7
ships with version 0.7.0 which lacks this API. Revert to using the older
versioning API if we detect an old libpsl version.

Follow-up to 72bd88adde0e8cf6e63644a7d6df1da01a399db4
Bug: https://curl.se/mail/archive-2024-02/0004.html
Reported-by: Scott Mutter
Closes #12872
2024-02-06 09:50:35 +01:00
Stefan Eissing
d6825df334
openssl-quic: check on Windows that socket conv to int is possible
Fixes #12861
Closes #12865
2024-02-05 23:45:01 +01:00
Daniel Stenberg
5cc2b016c3
md4: include strdup.h for the memdup proto
Reported-by: Erik Schnetter
Fixes #12849
Closes #12863
2024-02-05 23:38:12 +01:00
Stefan Eissing
0dc036225b
HTTP/2: write response directly
- use the new `Curl_xfer_write_resp()` to write incoming responses
  directly to the client
- eliminates `stream->recvbuf`
- memory consumption on parallel transfers minimized

Closes #12828
2024-02-05 22:29:21 +01:00
Faraz Fallahi
a84ad94dd6
connect.c: fix typo
Closes #12858
2024-02-05 19:33:14 +01:00
Daniel Stenberg
b8c003832d
sendf: ignore response body to HEAD
and mark the stream for close, but return OK since the response this far
was ok - if headers were received. Partly because this is what curl has
done traditionally.

Test 499 verifies. Updates test 689.

Reported-by: Sergey Bronnikov
Bug: https://curl.se/mail/lib-2024-02/0000.html
Closes #12842
2024-02-01 22:37:26 +01:00
Daniel Stenberg
10491957e3
ftp: treat a 226 arriving before data as a signal to read data
For active mode transfers.

Due to some interesting timing, curl can sometimes get the 226 (transfer
complete) over the control channel first, before the data connection
signals readability. If this happens, use that as a signal to check the
data connection.

Additionally, set the socket filter in listen mode *before* the
PORT/EPRT command is issued, to reduce the risk that the little time gap
could interfere.

This issue never reproduced for me on Debian and takes several hundred
rounds for me to trigger on my mac.

Reported-by: Stefan Eissing
Fixes #12823
Closes #12841
2024-02-01 22:32:38 +01:00
Daniel Stenberg
8243ad6458
asyn-thread: use wakeup_close to close the read descriptor
Reported-by: Dan Fandrich
Ref: #12834
Closes #12836
2024-02-01 13:42:48 +01:00
Stefan Eissing
e75a48d2c3
ntml_wb: fix buffer type typo
Closes #12825
2024-02-01 13:04:56 +01:00
Daniel Stenberg
011325ff29
lib: error out on multissl + http3
Since the QUIC/h3 code has no knowledge or handling of multissl it might
bring unintended consequences if we allow it.

configure, cmake and curl_setup.h all now reject this combination.

Assisted-by: Viktor Szakats
Assisted-by: Gisle Vanem
Ref: #12806
Closes #12807
2024-01-29 16:37:08 +01:00
Viktor Szakats
5b286c2508
build: delete/replace clang warning pragmas
- delete redundant warning suppressions for `-Wformat-nonliteral`.
  This now relies on `CURL_PRINTF()` and it's theoratically possible
  that this macro isn't active but the warning is. We're ignoring this
  as a corner-case here.

- replace two pragmas with code changes to avoid the warnings.

Follow-up to aee4ebe59161d0a5281743f96e7738ad97fe1cd4 #12803
Follow-up to 09230127589eccc7e01c1a7217787ef8e64f3328 #12540
Follow-up to 3829759bd042c03225ae862062560f568ba1a231 #12489

Reviewed-by: Daniel Stenberg
Closes #12812
2024-01-27 21:19:41 +00:00
Daniel Stenberg
066ed4e514
http: only act on 101 responses when they are HTTP/1.1
For 101 responses claiming to be any other protocol, bail out. This
would previously trigger an assert.

Add test 1704 to verify.

Bug: https://bugs.chromium.org/p/oss-fuzz/issues/detail?id=66184
Closes #12811
2024-01-27 21:49:20 +01:00
Stefan Eissing
bdae275958
mqtt, remove remaining use of data->state.buffer
Closes #12799
2024-01-27 12:42:19 +01:00
Daniel Stenberg
623c3a8fa0
x509asn1: switch from malloc to dynbuf
Closes #12808
2024-01-27 12:27:35 +01:00
Daniel Stenberg
cd48e8f8a9
x509asn1: make utf8asn1str() use dynbuf instead of malloc + memcpy
Closes #12808
2024-01-27 12:27:32 +01:00
Daniel Stenberg
98b41dd4bb
x509asn1: reduce malloc in Curl_extract_certinfo
Using dynbuf

Closes #12808
2024-01-27 12:27:25 +01:00
Daniel Stenberg
aee4ebe591
krb5: add prototype to silence clang warnings on mvsnprintf()
"error: format string is not a string literal"

Follow-up to 09230127589eccc7 which made the warning appear

Assisted-by: Viktor Szakats
Closes #12803
2024-01-26 23:28:28 +01:00
Daniel Stenberg
641257ea08
x509asn1: remove code for WANT_VERIFYHOST
No code ever sets this anymore since we dropped gskit

Follow-up to 78d6232f1f326b9ab4d

Closes #12804
2024-01-26 18:11:07 +01:00
Daniel Stenberg
a3a940702a
socks: reduce the buffer size to 600 (from 8K)
This is malloc'ed memory and it does not more. Test 742 helps us verify
this.

Closes #12789
2024-01-26 16:40:03 +01:00
Stefan Eissing
a6c9a33bf8
file+ftp: use stack buffers instead of data->state.buffer
Closes #12789
2024-01-26 16:38:21 +01:00
Stefan Eissing
9a90c9dd64
vtls: receive max buffer
- do not only receive one TLS record, but try to fill
  the passed buffer
- consider <4K remaning space is "filled".

Closes #12801
2024-01-26 15:46:15 +01:00
Patrick Monnerat
7b2d98dfad sasl: make login option string override http auth
- Use http authentication mechanisms as a default, not a preset.

Consider http authentication options which are mapped to SASL options as
a default (overriding the hardcoded default mask for the protocol) that
is ignored if a login option string is given.

Prior to this change, if some HTTP auth options were given, sasl mapped
http authentication options to sasl ones but merged them with the login
options.

That caused problems with the cli tool that sets the http login option
CURLAUTH_BEARER as a side-effect of --oauth2-bearer, because this flag
maps to more than one sasl mechanisms and the latter cannot be cleared
individually by the login options string.

New test 992 checks this.

Fixes https://github.com/curl/curl/issues/10259
Closes https://github.com/curl/curl/pull/12790
2024-01-26 02:58:21 -05:00
Stefan Eissing
65c7e4f92b
socks: use own buffer instead of data->state.buffer
Closes #12788
2024-01-26 08:50:17 +01:00
Daniel Stenberg
bc604619de
socks: fix generic output string to say SOCKS instead of SOCKS4
... since it was also logged for SOCKS5.

Closes #12797
2024-01-26 08:44:52 +01:00
Stefan Eissing
6d85228709
ssh: use stack scratch buffer for seeks
- instead of data->state.buffer

Closes #12794
2024-01-25 22:33:49 +01:00
Daniel Stenberg
31db4bfa73
krb5: access the response buffer correctly
As the pingpong code no longer uses the download buffer.

Folllow-up to c2d973627bab12ab
Pointed-out-by: Stefan Eissing
Closes #12796
2024-01-25 22:32:53 +01:00
Stefan Eissing
73c4aaa2c0
mqtt: use stack scratch buffer for recv+publish
- instead of data->state.buffer

Closes #12792
2024-01-25 22:32:18 +01:00
Stefan Eissing
0c05b8f813
telnet, use stack scratch buffer for do
- instead of data->state.buffer

Closes #12793
2024-01-25 22:31:36 +01:00
Stefan Eissing
a069c6aeb2
http, use stack scratch buffer
- instead of data->state.buffer

Closes #12791
2024-01-25 22:31:08 +01:00
Stefan Eissing
8e9c304f4a
ntlm_wb: do not use data->state.buf any longer
Closes #12787
2024-01-25 17:04:02 +01:00
Daniel Stenberg
879a44084f
http: check for "Host:" case insensitively
When checking if the user wants to replace the header, the check should
be case insensitive.

Adding test 461 to verify

Found-by: Dan Fandrich
Ref: #12782
Closes #12784
2024-01-25 13:53:09 +01:00
Daniel Stenberg
199c1d720d
http: remove comment reference to a removed solution
Follow-up to 58974d25d

Closes #12785
2024-01-25 11:26:23 +01:00
Daniel Stenberg
c2d973627b
pingpong: stop using the download buffer
The pingpong logic now uses its own dynbuf for receiving command
response data.

When the "final" response header for a commanad has been received, that
final line is left first in the recvbuf for the protocols to parse at
will. If there is additional data behind the final response line, the
'overflow' counter is indicate how many bytes.

Closes #12757
2024-01-25 08:34:23 +01:00
Viktor Szakats
a808aab068
cmake: rework options to enable curl and libcurl docs
Rework CMake options for building/using curl tool and libcurl manuals.

- rename `ENABLE_MANUAL` to `ENABLE_CURL_MANUAL`, meaning:
  to build man page and built-in manual for curl tool.

- rename `BUILD_DOCS` to `BUILD_LIBCURL_DOCS`, meaning:
  to build man pages for libcurl.

- `BUILD_LIBCURL_DOCS` now works without having to enable
  `ENABLE_CURL_MANUAL` too.

- drop support for existing CMake-level `USE_MANUAL` option to avoid
  confusion. (It used to work with the effect of current
  `ENABLE_CURL_MANUAL`, but only by accident.)

Assisted-by: Richard Levitte
Ref: #12771
Closes #12773
2024-01-24 23:22:27 +00:00
Daniel Stenberg
162113676a
urlapi: remove assert
This assert triggers wrongly when CURLU_GUESS_SCHEME and
CURLU_NO_AUTHORITY are both set and the URL is a single path.

I think this assert has played out its role. It was introduced in a
rather big refactor.

Follow-up to 4cfa5bcc9a

Reported-by: promptfuzz_ on hackerone
Closes #12775
2024-01-24 23:15:13 +01:00
Daniel Stenberg
e17b89db8f
osslq: remove the TLS library from the version output
Since we only support using a single TLS library at any one time, we
know that the TLS library for QUIC is the same that is also shown for
regular TLS.

Fixes #12763
Reported-by: Viktor Szakats
Closes #12767
2024-01-23 23:27:45 +01:00