Commit Graph

13999 Commits

Author SHA1 Message Date
Viktor Szakats
72abf7c13a
lib: tidy up types and casts
Cherry-picked from #13489
Closes #13862
2024-06-05 14:02:39 +02:00
Stefan Eissing
937ba94ed5
vtls: new io_need flags for poll handling
- decouple need to recv/send from negotiation state, we need
  this later in shutdown handling as well
- move ssl enums from urldata.h to vtls_int.h
- implement use of `connssl->io_need` in vtls.c. and all backends

Closes #13879
2024-06-05 09:03:38 +02:00
Daniel Stenberg
f75aa2857f
cfilters: make Curl_conn_connect always assign 'done'
It could return error without assigning it, and we have a caller in
multi.c that assumes it gets set.

Spotted by CodeSonar
Closes #13884
2024-06-05 08:51:14 +02:00
Daniel Stenberg
b049388d47
url: allow DoH transfers to override max connection limit
When reaching the set maximum limit of allowed connections, allow a new
connection anyway if the transfer is created for the (internal) purpose
of doing a DoH name resolve. Otherwise, unrelated "normal" transfers can
starve out new DoH requests making it impossible to name resolve for new
transfers.

Bug: https://curl.se/mail/lib-2024-06/0001.html
Reported-by: kartatz
Closes #13880
2024-06-05 08:35:12 +02:00
Viktor Szakats
998b17ea7f
windows: fix UWP builds, add GHA job
Add new job to test building for UWP (aka `CURL_WINDOWS_APP`).

Fix fallouts when building for UWP:
- rand: do not use `BCryptGenRandom()`.
- cmake: disable using win32 LDAP.
- cmake: disable telnet.
- version_win32: fix code before declaration.
- schannel: disable `HAS_MANUAL_VERIFY_API`.
- schannel: disable `SSLSUPP_PINNEDPUBKEY`
  and make `schannel_checksum()` a stub.
  Ref: e178fbd40a #1429
- schannel: make `cert_get_name_string()` a failing stub.
- system_win32: make `Curl_win32_impersonating()` a failing stub.
- system_win32: try to fix `Curl_win32_init()` (untested).
- threads: fix to use `CreateThread()`.
- src: disable searching `PATH` for the CA bundle.
- src: disable bold text support and capability detection.
- src: disable `getfiletime()`/`setfiletime()`.
- tests: make `win32_load_system_library()` a failing stub.
- tests/server/util: make it compile.
- tests/server/sockfilt: make it compile.
- tests/lib3026: fix to use `CreateThread()`.

See individual commits for build error details.

Some of these fixes may have better solutions, and some may not work
as expected. The goal of this patch is to make curl build for UWP.

Closes #13870
2024-06-05 00:52:24 +02:00
Orgad Shaneh
3060557af7
socket: support binding to interface *AND* IP
Introduce new notation for CURLOPT_INTERFACE / --interface:
ifhost!<interface>!<host>

Binding to an interface doesn't set the address, and an interface can
have multiple addresses.

When binding to an address (without interface), the kernel is free to
choose the route, and it can route through any device that can access
the target address, not necessarily the one with the chosen address.

Moreover, it is possible for different interfaces to have the same IP
address, on which case we need to provide a way to be more specific.

Factor out the parsing part of interface option, and add unit tests:
1663.

Closes #13719
2024-06-04 23:47:54 +02:00
Andy Pan
23fe1a52dc
socketpair: add eventfd and use SOCK_NONBLOCK for socketpair()
Currently, we use `pipe` for `wakeup_create`, which requires ***two***
file descriptors. Furthermore, given its complexity inside, `pipe` is a
bit heavyweight for just a simple event wait/notify mechanism.

`eventfd` would be a more suitable solution for this kind of scenario,
kernel also advocates for developers to use `eventfd` instead of `pipe`
in some simple use cases:

    Applications can use an eventfd file descriptor instead of a pipe
    (see pipe(2) in all cases where a pipe is used simply to signal
    events. The kernel overhead of an eventfd file descriptor is much
    lower than that of a pipe, and only one file descriptor is required
    (versus the two required for a pipe).

This change adds the new backend of `eventfd` for `wakeup_create` and
uses it where available, eliminating the overhead of `pipe`. Also, it
optimizes the `wakeup_create` to eliminate the system calls that make
file descriptors non-blocking by moving the logic of setting
non-blocking flags on file descriptors to `socketpair.c` and using
`SOCK_NONBLOCK` for `socketpair(2)`, `EFD_NONBLOCK` for `eventfd(2)`.

Ref:
https://man7.org/linux/man-pages/man7/pipe.7.html
https://man7.org/linux/man-pages/man2/eventfd.2.html
https://man7.org/linux/man-pages/man2/socketpair.2.html
https://www.gnu.org/software/gnulib/manual/html_node/eventfd.html

Closes #13874
2024-06-04 23:45:36 +02:00
Viktor Szakats
83384669ef
lib: fix thread entry point to return DWORD on WinCE
We already do this in `tests/server/util.c`:
97e5e37cc8/tests/server/util.c (L604-L606)
and in `sockfilt.c`, `lib3026.c`.

Before this patch it returned `unsigned int`.

Closes #13877
2024-06-04 18:47:18 +02:00
Andy Pan
3392f0f97e
socket: use SOCK_NONBLOCK to eliminate extra system call
Every time function `cf_socket_open()` is called to create a socket,
`curlx_nonblock()` is called to make that socket non-blocking. And
`curlx_nonblock()` will cost us 1 or 2 system calls (2 for `fcntl()`, 1
for `ioctl()`, etc.), meanwhile, tucking `SOCK_NONBLOCK` and
`SOCK_CLOEXEC` into the `type` argument for `socket()` is widely
supported across UNIX-like OS: Linux, *BSD, Solaris, etc. With that
ability, we can save 1 or 2 system calls on each socket.

Another change in this PR is to eliminate the redundant
`curlx_nonblock()` call on the socket in `cf_udp_setup_quic()` as that
socket created by `cf_socket_open()` is already non-blocking.

Ref:
https://man7.org/linux/man-pages/man2/socket.2.html
https://man.freebsd.org/cgi/man.cgi?socket(2)
https://man.dragonflybsd.org/?command=socket&section=2
https://man.netbsd.org/socket.2
https://man.openbsd.org/socket
https://docs.oracle.com/cd/E88353_01/html/E37843/socket-3c.html
https://illumos.org/man/3SOCKET/socket
...

Closes #13855
2024-06-04 17:51:28 +02:00
Stefan Eissing
5f9017d4e2
mbedtls: v3.6.0 workarounds
- add special sauce to disable unwanted peer verification by mbedtls
  when negotiating TLS v1.3
- add special sauce for MBEDTLS_ERR_SSL_RECEIVED_NEW_SESSION_TICKET
  return code on *writing* TLS data. We assume the data had not been
  written and EAGAIN.
- return correct Curl error code when peer verification failed.
- disable test_08_05 with 50 HTTP/1.1 connections, as mbedtls reports a
  memory allocation failed during handshake.
- bump CI mbedtls version to 3.6.0

Fixes #13653
Closes #13838
2024-06-04 09:02:37 +02:00
Stefan Eissing
5dd8f13bfc
gnutls: support CA caching
- similar to openssl, use a shared 'credentials' instance
  among TLS connections with a plain configuration.
- different to openssl, a connection with a client certificate
  is not eligible to sharing.
- document CURLOPT_CA_CACHE_TIMEOUT in man page

Closes #13795
2024-06-04 08:17:55 +02:00
Andy Pan
f51fa8f169
tcpkeepalive: support setting TCP keep-alive parameters on Solaris <11.4
Solaris didn't support TCP_KEEPIDLE and TCP_KEEPINTVL until 11.4,
before that it use TCP_KEEPALIVE_THRESHOLD and TCP_KEEPALIVE_ABORT_THRESHOLD
as the substitute. Therefore, for Solaris <11.4 we need to use this substitute
for setting TCP keep-alive parameters.

Ref:
https://docs.oracle.com/cd/E86824_01/html/E54777/tcp-7p.html
https://docs.oracle.com/cd/E88353_01/html/E37851/tcp-4p.html

Closes #13864
2024-06-03 23:04:05 +02:00
Viktor Szakats
0887297100
lib/v*: tidy up types and casts
Also add a couple of negative checks.

Cherry-picked from #13489
Closes #13622
2024-06-02 19:27:17 +02:00
Bo Anderson
eacec9a350
x509asn1: add some common ECDSA OIDs
Closes #13857
2024-06-02 17:30:09 +02:00
Bo Anderson
9aa1d412b8
x509asn1: fallback to dotted OID representation
Reported-by: Luke Hamburg
Fixes #13845
Closes #13858
2024-06-02 17:23:50 +02:00
MonkeybreadSoftware
810933d779
vtls: deprioritize Secure Transport
Moved Secure Transport behind OpenSSL, so we can build CURL with both
and prefer using OpenSSL over Secure Transport by default.

Closes #13547
2024-06-02 17:10:17 +02:00
Daniel Stenberg
655d44d139
urlapi: add CURLU_NO_GUESS_SCHEME
Used for extracting:

- when used asking for a scheme, it will return CURLUE_NO_SCHEME if the
  stored information was a guess

- when used asking for a URL, the URL is returned without a scheme, like
  when previously given to the URL parser when it was asked to guess

- as soon as the scheme is set explicitly, it is no longer internally
  marked as guessed

The idea being:

1. allow a user to figure out if a URL's scheme was set as a result of
  guessing

2. extract the URL without a guessed scheme

3. this makes it work similar to how we already deal with port numbers

Extend test 1560 to verify.

Closes #13616
2024-06-01 23:51:42 +02:00
Daniel Stenberg
80aa519545
wolfssl: support CA caching
As a bonus, add SSLSUPP_CA_CACHE to let TLS backends signal its support
for this so that *setopt() return error if there is no support.

Closes #13786
2024-06-01 23:50:36 +02:00
Andy Pan
6da320357f
socket: change TCP keepalive from ms to seconds on DragonFly BSD
DragonFly BSD changed the time unit for TCP keep-alive from milliseconds
to seconds since v5.8, thus setting the keepalive options with
milliseconds with curl/libcurl will result in unexpected behaviors on
DragonFlyBSD 5.8+

Distinguish the DragonFly BSD versions and use the proper time units
accordingly.

Ref:
  https://lists.dragonflybsd.org/pipermail/commits/2019-July/719125.html
  965b380e96/sys/sys/param.h (L207)

Fixes #13847
Closes #13848
2024-06-01 23:46:41 +02:00
Daniel Stenberg
e78913e6b5
noproxy: patterns need to be comma separated
or they will not parse correctly.

Mentioned in DEPRECATED since Janurary 2023 (in 7ad8a7ba9e).

Closes #13789
2024-06-01 12:25:13 +02:00
Jan Venekamp
4e2c45110c
sectransp: remove large cipher table
Previously a large table of ciphers was used to determine the default
ciphers and to lookup manually selected ciphers names.

With the lookup of the manually selected cipher names moved to
Curl_cipher_suite_walk_str() the large table is no longer needed for
that purpose.

The list of manually selected cipher can now be intersected with the
ciphers supported by Secure Transport (SSLGetSupportedCiphers()),
instead of using the fixed table for that.

The other use of the table was to filter the list of all supported
ciphers offered by Secure Transport to create a list of ciphers to
use by default, excluding ciphers in the table marked as weak.

Instead of using a complement based approach (exclude weak), switch
to using an intersection with a smaller list of ciphers deemed
appropriate.

Closes #13823
2024-06-01 11:00:06 +02:00
Stephen Farrell
48292d8c93
openSSL: fix hostname handling when using ECH
Reported-by: vvb2060
Fixes #13818
Closes #13822
2024-06-01 10:48:23 +02:00
Jay Satiro
acb9effcdd cf-socket: improve SO_SNDBUF update for Winsock
- Rename: Curl_sndbufset => Curl_sndbuf_init

- Rename: win_update_buffer_size => win_update_sndbuf_size

- Save the last set SO_SNDBUF size to compare against so that we can
  avoid setsockopt calls every second.

This is a follow-up to 0b520e12 which moved the SO_SNDBUF update check
into cf-socket. This change improves it further by making the function
names easier to understand and reducing the amount of setsockopt calls.

Closes https://github.com/curl/curl/pull/13827
2024-05-30 13:56:37 -04:00
Stefan Eissing
c8096668ae
multi: fix multi_wait() timeout handling
- determine the actual poll timeout *after* all sockets
  have been collected. Protocols and connection filters may
  install new timeouts during collection.
- add debug logging to test1533 where the mistake was noticed

Reported-by: Matt Jolly
Fixes #13782
Closes #13825
2024-05-30 08:29:00 +02:00
Viktor Szakats
3b9569c3e2
lib: prefer var = time(NULL) over time(&var)
Following up on previous occurrences showing up as gcc warnings, replace
the remaining `time(&var)` calls with `var = time(NULL)`, though these
aren't specifically causing compiler warnings. These are in the TFTP
client code (`lib/tftp.c`), except one which is in a debug branch in
`lib/http_aws_sigv4.c`.

What's unexplainable is that this patch seems to mitigate TFTP tests
often hanging or going into an infinite loop on GHA windows workflows
with MSYS2, mingw-w64 and MSVC (Cygwin is unaffected):
  https://github.com/curl/curl/pull/13599#issuecomment-2119372376
TFTP hangs did not entirely disappear though, so could be unrelated.

`time()` docs:
https://learn.microsoft.com/en-us/cpp/c-runtime-library/reference/time-time32-time64
https://manpages.debian.org/bookworm/manpages-dev/time.2.en.html

Follow-up to 58ca0a2f07 #13800
Follow-up to d0728c9109 #13643
Closes #13815
2024-05-29 21:45:28 +02:00
Stefan Eissing
0b520e1250 winsock: move SO_SNDBUF update into cf-socket
- Move the code that updates the SO_SNDBUF size for Windows to
  cf_socket_send.

Prior to this change the code was in readwrite_upload but the socket
filter is the more appropriate place because it applies to all sends.

Background:

For Windows users SO_SNDBUF (the total per-socket buffer size reserved
by Winsock for sends) is updated dynamically by libcurl during the
transfer. This is because Windows does not do it automatically for
non-blocking sockets and without it the performance of large transfers
may suffer.

Closes https://github.com/curl/curl/pull/13763
2024-05-29 13:23:15 -04:00
Jan Venekamp
2b52fe4115
sectransp: use common code for cipher suite lookup
Take advantage of the Curl_cipher_suite_walk_str() and
Curl_cipher_suite_get_str() functions introduced in commit fba9afe.

Closes #13521
2024-05-29 13:08:14 +02:00
Matthias Gatto
768909d89c
aws-sigv4: url encode the canonical path
Refactors canon_query, so it could use the encoding part of the function
to use it in the path.

As the path doesn't encode '/', but encode '=', I had to add some
conditions to know If I was doing the query or path encoding.

Also, instead of adding a `bool in_path` variable, I use `bool
*found_equals` to know if the function was called for the query or path,
as found_equals is used only in query_encoding.

Test 472 verifies.

Reported-by: Alexander Shtuchkin
Fixes #13754
Closes #13814

Signed-off-by: Matthias Gatto <matthias.gatto@outscale.com>
2024-05-29 13:02:52 +02:00
vvb2060
f284289720
openssl: fix %-specifier in infof() call
Closes #13816
2024-05-28 23:14:24 +02:00
Daniel Stenberg
582743f2e7
openssl/gnutls: rectify the TLS version checks for QUIC
The versions check wrongly complained and return error if the *minimum*
version was set to something less than 1.3. QUIC is always TLS 1.3, but
that means minimum 1.2 is still fine to ask for.

This also renames the local variable to make the mistake harder to make
in the future.

Regression shipped in 8.8.0

Follow-up to 3210101088

Reported-by: fds242 on github
Fixes #13799
Closes #13802
2024-05-28 10:39:02 +02:00
Stefan Eissing
7bbad0c033
gnutls: improve TLS shutdown
local ftp upload tests sometimes failed with an invalid TLS record being
reported by gnutls. vsftp did log that the shutdown was not regarded as
clean, failing the control connection thereafter.

These changes make test_31_05 work reliable locally.

- on closing the SSL filter, shutdown READ *and* WRITE
- on closing, try a receive after shutdown is sent
- convert to DEBUGF to CURL_TRC_CF

Closes #13790
2024-05-28 10:37:52 +02:00
Viktor Szakats
59dc9f7e69
build: untangle CURLDEBUG and DEBUGBUILD macros
`CURLDEBUG` is meant to enable memory tracking, but in a bunch of cases,
it was protecting debug features that were supposed to be guarded with
`DEBUGBUILD`.

Replace these uses with `DEBUGBUILD`.

This leaves `CURLDEBUG` uses solely for its intended  purpose: to enable
the memory tracking debug feature.

Also:
- autotools: rely on `DEBUGBUILD` to enable `checksrc`.
  Instead of `CURLDEBUG`, which worked in most cases because debug
  builds enable `CURLDEBUG` by default, but it's not accurate.
- include `lib/easyif.h` instead of keeping a copy of a declaration.
- add CI test jobs for the build issues discovered.

Ref: https://github.com/curl/curl/pull/13694#issuecomment-2120311894
Closes #13718
2024-05-28 08:12:00 +02:00
Viktor Szakats
0fd794df35
cmake: fix -Wredundant-decls in unity/mingw-w64/gcc/curldebug/DLL builds
It affected cmake-unity shared-curltool curldebug mingw-w64 gcc builds
when building the `testdeps` target.

Apply the solution already used in `lib/base64.c` and `lib/dynbuf.c`
to fix it.

Also update an existing GHA CI job to test the issue fixed.

```
In file included from curl/lib/version_win32.c:35,
                 from curl/_bld/src/CMakeFiles/curl.dir/Unity/unity_0_c.c:145:
curl/lib/memdebug.h:52:14: error: redundant redeclaration of 'curl_dbg_logfile' [-Werror=redundant-decls]
   52 | extern FILE *curl_dbg_logfile;
      |              ^~~~~~~~~~~~~~~~
In file included from curl/src/slist_wc.c:32,
                 from curl/_bld/src/CMakeFiles/curl.dir/Unity/unity_0_c.c:4:
curl/lib/memdebug.h:52:14: note: previous declaration of 'curl_dbg_logfile' with type 'FILE *' {aka 'struct _iobuf *'}
   52 | extern FILE *curl_dbg_logfile;
      |              ^~~~~~~~~~~~~~~~
curl/lib/memdebug.h:55:44: error: redundant redeclaration of 'curl_dbg_malloc' [-Werror=redundant-decls]
   55 | CURL_EXTERN ALLOC_FUNC ALLOC_SIZE(1) void *curl_dbg_malloc(size_t size,
      |                                            ^~~~~~~~~~~~~~~
curl/lib/memdebug.h:55:44: note: previous declaration of 'curl_dbg_malloc' with type 'void *(size_t,  int,  const char *)' {aka 'void *(long long unsigned int,  int,  const char *)'}
   55 | CURL_EXTERN ALLOC_FUNC ALLOC_SIZE(1) void *curl_dbg_malloc(size_t size,
      |                                            ^~~~~~~~~~~~~~~
[...]
curl/lib/memdebug.h:110:17: error: redundant redeclaration of 'curl_dbg_fclose' [-Werror=redundant-decls]
  110 | CURL_EXTERN int curl_dbg_fclose(FILE *file, int line, const char *source);
      |                 ^~~~~~~~~~~~~~~
curl/lib/memdebug.h:110:17: note: previous declaration of 'curl_dbg_fclose' with type 'int(FILE *, int,  const char *)' {aka 'int(struct _iobuf *, int,  const char *)'}
  110 | CURL_EXTERN int curl_dbg_fclose(FILE *file, int line, const char *source);
      |                 ^~~~~~~~~~~~~~~
```
Ref: https://ci.appveyor.com/project/curlorg/curl/builds/49840554/job/a4aoet17e9qnqx1a#L362

After: https://ci.appveyor.com/project/curlorg/curl/builds/49843735/job/hbo2uah2vj0ns523

Ref: #13689 (CI testing this PR with `DEBUGBUILD`/`CURLDEBUG`/shared-static combinations)
Depends-on: #13694
Depends-on: #13800
Closes #13705
2024-05-27 22:56:13 +02:00
Viktor Szakats
58ca0a2f07
lib: fix gcc warning in certain debug builds
```
curl/lib/http_aws_sigv4.c:536:10: error: 'clock' may be used uninitialized [-Werror=maybe-uninitialized]
  536 |   time_t clock;
      |          ^~~~~
```
Ref: https://github.com/curl/curl/actions/runs/9158755123/job/25177765000#step:13:79

Cherry-picked from #13718
Closes #13800
2024-05-27 22:26:45 +02:00
Viktor Szakats
fc8e0dee30
build: untangle UNITTESTS and DEBUGBUILD macros
- fix `DEBUGBUILD` guards that should be `UNITTESTS`, in libcurl code
  used by unit tests.
- fix guards for libcurl functions used in unit tests only.
- sync `UNITTEST` attribute between declarations and definitions.
- drop `DEBUGBUILD` guard from test `unit2600`.
- fix guards for libcurl HSTS code used by both a unit test (`unit1660`)
  and `test0446`.
- update an existing AppVeyor CI job to test the issues fixed.

This fixes building tests with `CURLDEBUG` enabled but `DEBUGBUILD`
disabled. This can happen when building tests with CMake with
`ENABLE_DEBUG=ON` in Release config, or with `ENABLE_CURLDEBUG=ON`
and _without_ `ENABLE_DEBUG=ON`. Possibly also with autotools
when using `--enable-curldebug` without `--enable-debug`.

Test results:
- before:
  https://ci.appveyor.com/project/curlorg/curl/builds/49835609
  https://ci.appveyor.com/project/curlorg/curl/builds/49898529/job/k8qpbs8idby70smw
  https://github.com/curl/curl/actions/runs/9259078835/job/25470318167?pr=13798#step:13:821
- after: https://ci.appveyor.com/project/curlorg/curl/builds/49839255
  (the two failures are unrelated, subject to PR #13705)

Ref: #13592 (issue discovery)
Ref: #13689 (CI testing this PR with `DEBUGBUILD`/`CURLDEBUG` combinations)
Closes #13694
2024-05-27 21:15:50 +02:00
Viktor Szakats
739ef9804d
cmake: use APPLE instead of CMAKE_SYSTEM_NAME string
Follow-up to a86254b393 #12515
Closes #13713
2024-05-27 18:17:43 +02:00
Viktor Szakats
0e176cabe4
cmake: whitespace, formatting/tidy-up in comments
Also correct casing in a few option descriptions.

Closes #13711
2024-05-27 18:07:10 +02:00
Daniel Stenberg
4157ccb88c
libssh: remove CURLOPT_SSL_VERIFYHOST check
It was never meant for SSH: it should rely on the knownhosts file (if
set) in the same way libssh2 already does.

Reported-by: James Abbatiello
Fixes #13767
Closes #13781
2024-05-26 12:01:16 +02:00
Stefan Eissing
e101a7a8b0
multi: add multi->proto_hash, a key-value store for protocol data
- add `Curl_hash_add2()` that passes a destructor function for
  the element added. Call element destructor instead of hash
  destructor if present.
- multi: add `proto_hash` for protocol related information,
  remove `struct multi_ssl_backend_data`.
- openssl: use multi->proto_hash to keep x509 shared store
- schannel: use multi->proto_hash to keep x509 shared store
- vtls: remove Curl_free_multi_ssl_backend_data() and its
  equivalents in the TLS backends

Closes #13345
2024-05-26 00:15:01 +02:00
Stefan Eissing
17af2bca58
http: write last header line late
- HEADERFUNCTIONS might inspect response properties like
  CURLINFO_CONTENT_LENGTH_DOWNLOAD_T on seeing the last header line. If
  the line is being written before this is initialized, values are not
  available.

- write the last header line late when analyzing a HTTP response so that
  all information is available at the time of the writing.

- add test1485 to verify that CURLINFO_CONTENT_LENGTH_DOWNLOAD_T works
  on seeing the last header.

Fixes #13752
Reported-by: Harry Sintonen
Closes #13757
2024-05-25 23:42:53 +02:00
Jay Satiro
02b14378e6 openssl: stop duplicate ssl key logging for legacy OpenSSL
- Don't call the keylog function if it has already logged the key.

For old OpenSSL versions and its forks that do not have support for
OpenSSL's keylog callback, libcurl has its own legacy key logging
function that logs the TLS 1.2 (and earlier) key (client random + master
key) on a single line.

Prior to this change, since e7de80e8 (precedes 8.8.0), the legacy key
logging function could write the same key line more than once (usually
twice) due to some incorrect logic.

Closes https://github.com/curl/curl/pull/13683
2024-05-24 15:22:53 -04:00
Stefan Eissing
28284c8f33
transfer: remove curl_upload_refill_watermark, no longer used
the define applied to upload buffers which we removed

Closes #13764
2024-05-24 14:54:07 +02:00
Pavel P
30c3bf1656 cmake: fix building in unity mode
- Fix sha256 and sha512 duplicate macro names (eg function-like macro Ch
  is now Sha256_Ch and Sha512_Ch).

- Avoid defining short defines like R, S. (eg S is now Sha256_S).

Closes https://github.com/curl/curl/pull/13751
2024-05-24 03:25:19 -04:00
Pavel P
0caadc1f24
asyn-thread: avoid using GetAddrInfoExW with impersonation
Multiple reports suggest that GetAddrInfoExW fails when impersonation is
used. This PR checks if thread is impersonating and avoids using
GetAddrInfoExW api.

Reported-by: Keerthi Timmaraju
Assisted-by: edmcln on github
Fixes #13612
Closes #13738
2024-05-23 23:56:58 +02:00
Stefan Eissing
30de937bda
transfer: conn close on paused upload
- add 2 variations on test_07_42 which PAUSEs uploads
  and response connections terminating either right away
  or after the 100-continue response
- when detecting the connection being closed in transfer.c
  readwrite_data(), clear ALL send bits in data->req.keepon.
  It no longer makes send to wait for a KEEP_SEND_PAUSE or HOLD.
- in the protocol client writer add the check for incomplete
  response bodies. When an EOS is seen and the length is known,
  check that and fail if bytes are missing.

Reported-by: Sergey Bronnikov
Fixes #13740
Closes #13750
2024-05-23 23:55:09 +02:00
Stefan Eissing
1c4813c769
mbedtls: send close-notify on close
- send the TLS close notify message when cloding down
  the mbedtls connection filter
- this is a "least" effort version and, as other TLS filters,
  is lacking a graceful send/receive/timeout for a really
  clean shutdown.

Closes #13745
2024-05-22 23:07:22 +02:00
Stefan Eissing
0c4b4c1e93
mbedtls: check version for cipher id
mbedtls_ssl_get_ciphersuite_id_from_ssl() seems to have been added in
mbedtls 3.2.0. Check for that version.

Closes #13749
2024-05-22 23:05:41 +02:00
Viktor Szakats
ca8cb245e1
cmake: fix building with both md4 and md5 in unity mode
Macro and static function names were colliding between
`lib/md4.c` and
`lib/md5.c`.

Fix it by namespacing these symbols.

Seen with a basic macOS build using these options:
`-DCMAKE_UNITY_BUILD=ON -DCURL_USE_SECTRANSP=ON`

Closes #13737
2024-05-22 14:16:40 +02:00
Stefan Eissing
f867942511
test: add test1546, chunked not last transfer encoding
with more than one transfer-encoding, 'chunked' must be the last added
to the writer stack (and therefore the first to decode). RFC 9112, ch.
6.1.

Closes #13736
2024-05-22 09:11:13 +02:00
Viktor Szakats
d13a3a6635
(lib)curl.rc: set debug flag also for CURLDEBUG and UNITTESTS
These macros also enable debug features in both libcurl and curl.
Enable `VS_FF_DEBUG` version resource flag when they are set.

Closes #13730
2024-05-22 08:38:12 +02:00
Nathan Moinvaziri
6ea9388157
url: remove duplicate call to Curl_conncache_remove_conn when pruning
- remove unnecessary prunedead struct from prune_dead_connections
- rename extract_if_dead to prune_if_dead for clarity

Closes #13710
2024-05-21 16:25:21 +02:00
Joseph Chen
c56071f41f
curl_setup.h: add support for IAR compiler
Closes #13728
2024-05-21 15:10:23 +02:00
Viktor Szakats
3b9ac3646b
hash: delete unused debug function
It had no use in the curl codebase and was also protected by the macro
`AGGRESSIVE_TEST` (renamed in 2020), also with no local reference.

Added in ca6e770837 (2002-11-11)

Closes #13729
2024-05-21 15:08:04 +02:00
Stefan Eissing
1d7b86e72b
content_encoding: reject transfer-encoding after chunked
reject a response that applies a transfer-encoding after a 'chunked'
encoding. RFC 9112 ch. 6.1 required chunked to be the final encoding.

Closes #13733
2024-05-21 15:06:41 +02:00
Stefan Eissing
5a4769b6d5
http: HEAD response body tolerance
- as reported in #13725, some servers wrongly send body bytes in
  responses to a HEAD request. This used to be tolerated in curl
  8.4 and before and leads to failed transfers in newer versions.
- restore previous behaviour for HTTP/1.1 and HTTP/2:
  * 1.1: do not add 'Transfer-Encoding' writers from HEAD
    responses. RFC 9112 says they do not apply.
  * 2: when the transfer expects 'no_body', to not report stream
    resets as error when all response headers have been received.

Reported-by: Jeroen Ooms
Fixes #13725
Closes #13732
2024-05-21 14:51:11 +02:00
Jay Satiro
9aae9bf817 openssl: revert keylog_callback support for LibreSSL
- Revert to the legacy TLS 1.2 key logging code for LibreSSL.

- Document SSLKEYLOGFILE for LibreSSL is TLS 1.2 max.

Prior to this change if the user specified a filename in the
SSLKEYLOGFILE environment variable and was using LibreSSL 3.5.0+ then
an empty file would be created and no keys would be logged.

This is effectively a revert of e43474b4 which changed openssl.c to use
SSL_CTX_set_keylog_callback for LibreSSL 3.5.0+. Unfortunately LibreSSL
added that function only as a stub that doesn't actually do anything.

Reported-by: Gonçalo Carvalho

Fixes https://github.com/curl/curl/issues/13672
Closes https://github.com/curl/curl/pull/13682
2024-05-20 03:55:40 -04:00
Viktor Szakats
0b31c713a9
cmake: merge two if(BUILD_TESTING) branches
Closes #13708
2024-05-19 23:18:20 +02:00
Stefan Eissing
afffd4c512
rustls: fix handshake done handling
- rustls report it has finished the TLS handshake *before*
  all relevant data has been sent off, e.g. it FINISHED message
- On connections the send data immediately, this was never noticed
  as the FINISHED in rustls buffers was send with the app data
- On passive FTP connections, curl does not send any data after
  the handshake, leaving FINISHED unsent and the server never
  responded as it was waiting on this.

Closes #13686
2024-05-17 16:51:45 +02:00
Daniel Stenberg
13ca4386ba
x509asn1: return error on missing OID
to avoid crash when dereferencing a NULL pointer.

Reported-by: Trzik on github
Patch-by: Trzik on github
Fixes #13684
Closes #13685
2024-05-17 14:54:41 +02:00
Viktor Szakats
d0728c9109
lib: fix compiler warnings (gcc)
Seen when setting `ENABLE_DEBUG=ON` and `-DDEBUGBUILD` for mingw-w64
gcc 13.2.0 CMake unity builds in 'Release' configurations.

```
curl/lib/curl_gethostname.c:71:5: error: 'strncpy' specified bound 1025 equals destination size [-Werror=stringop-truncation]
   71 |     strncpy(name, force_hostname, namelen);
      |     ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
In file included from curl/_bld/lib/CMakeFiles/libcurl_object.dir/Unity/unity_0_c.c:175:
In function 'hostcache_timestamp_remove',
    inlined from 'Curl_hash_clean_with_criterium' at curl/lib/hash.c:265:19,
    inlined from 'Curl_hash_clean_with_criterium' at curl/lib/hash.c:247:1,
    inlined from 'hostcache_prune' at curl/lib/hostip.c:228:3,
    inlined from 'Curl_hostcache_prune' at curl/lib/hostip.c:256:21:
curl/lib/hostip.c:205:12: error: 'now' may be used uninitialized [-Werror=maybe-uninitialized]
  205 |     time_t age = prune->now - c->timestamp;
      |            ^~~
curl/lib/hostip.c: In function 'Curl_hostcache_prune':
curl/lib/hostip.c:241:10: note: 'now' was declared here
  241 |   time_t now;
      |          ^~~
In function 'hostcache_timestamp_remove',
    inlined from 'fetch_addr' at curl/lib/hostip.c:310:8:
curl/lib/hostip.c:205:23: error: 'user.now' may be used uninitialized [-Werror=maybe-uninitialized]
  205 |     time_t age = prune->now - c->timestamp;
      |                  ~~~~~^~~~~
curl/lib/hostip.c: In function 'fetch_addr':
curl/lib/hostip.c:304:33: note: 'user' declared here
  304 |     struct hostcache_prune_data user;
      |                                 ^~~~
In file included from curl/_bld/lib/CMakeFiles/libcurl_object.dir/Unity/unity_0_c.c:40:
curl/lib/cf-socket.c: In function 'cf_socket_send':
curl/lib/cf-socket.c:1294:10: error: 'c' may be used uninitialized [-Werror=maybe-uninitialized]
 1294 |     if(c >= ((100-ctx->wblock_percent)*256/100)) {
      |        ~~^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
curl/lib/cf-socket.c:1292:19: note: 'c' was declared here
 1292 |     unsigned char c;
      |                   ^
In file included from curl/_bld/lib/CMakeFiles/libcurl_object.dir/Unity/unity_0_c.c:364:
In function 'tftp_state_timeout',
    inlined from 'tftp_multi_statemach' at curl/lib/tftp.c:1230:27:
curl/lib/tftp.c:1208:5: error: 'current' may be used uninitialized [-Werror=maybe-uninitialized]
 1208 |   if(current > state->rx_time + state->retry_time) {
      |     ^
curl/lib/tftp.c: In function 'tftp_multi_statemach':
curl/lib/tftp.c:1192:10: note: 'current' was declared here
 1192 |   time_t current;
      |          ^~~~~~~
```
Ref: https://ci.appveyor.com/project/curlorg/curl/builds/49792835/job/91c8dj5qb36spfe0#L112
Ref: https://github.com/curl/curl/actions/runs/9082968838/job/24960616145#step:12:62

Ref: #13592
Closes #13643
2024-05-17 12:00:19 +02:00
Andrew
fd0d2ed74a
wakeup_create: use FD_CLOEXEC/SOCK_CLOEXEC
for `pipe()`/`socketpair()`

Fixes #13618
Closes #13625
2024-05-16 23:02:12 +02:00
Stefan Eissing
6eee810db4
rustls: fix partial send handling
When TLS bytes could not completely sent off, the amount of plain bytes
already added to rustls were forgotten. This lead to send those byte
duplicate, corrupting the request send to the server.

Closes #13676
2024-05-16 23:01:15 +02:00
Juliusz Sosinowicz
4c46e277b2
vquic-tls: use correct cert name check API for wolfSSL
wolfSSL_X509_check_host checks the peer name against the alt names and
the common name.

Fixes #13487
Closes #13680
2024-05-16 22:44:29 +02:00
Antoine Bollengier
7c5ad6d278
socketpair: fix compilation when USE_UNIX_SOCKETS is not defined
Closes #13666
2024-05-16 10:50:02 +02:00
Stefan Eissing
df4a8c7661
rustsls: fix error code on receive
- use CURLE_RECV_ERROR instead of CURLE_READ_ERROR when receiving
  data fails.

Closes #13670
2024-05-16 10:48:07 +02:00
Orgad Shaneh
17fbed2008
socket: remove redundant call to getsockname
The result "add" is unused.

Closes #13655
2024-05-15 13:26:15 +02:00
Viktor Szakats
0c890d93eb
warnless: delete orphan declarations
Follow-up to 358f7e7577 #11932
Closes #13639
2024-05-14 16:37:11 +02:00
Daniel Gustafsson
d84a95de11 hsts: explicitly skip blank lines
Keep blank lines or lines containing only whitespace to make it all
the way to the more expensive sscanf call in hsts_add.

Closes: #13603
Reviewed-by: Daniel Stenberg <daniel@haxx.se>
2024-05-14 10:19:41 +02:00
Daniel Stenberg
aef369867f
lib: call Curl_strntolower instead of doing crafted loops
Closes #13627
2024-05-14 08:00:19 +02:00
Daniel Stenberg
9d6d614264
setopt: acknowledge errors proper for CURLOPT_COOKIEJAR
Error out on error, do not continue.

Closes #13624
2024-05-14 07:59:43 +02:00
Daniel Stenberg
6080805daa
vtls: remove duplicate assign
Curl_ssl_peer_cleanup() already clears the ->sni field, no point in
assigning it again.

Spotted by CodeSonar

Closes #13626
2024-05-14 07:58:59 +02:00
Daniel Stenberg
742196bd62
vauth: make two functions void that always just returned OK
Removes the need to check return values when they can never fail.

Pointed out by CodeSonar

Closes #13621
2024-05-13 22:38:25 +02:00
Daniel Stenberg
a11774776f
setopt: remove check for 'option' that is always true
- make sure that passing in option set to NULL clears the fields
  correctly

- remove the weird second take if Curl_parse_login_details() returns
  error

Follow-up to 7333faf00b

Spotted by CodeSonar

Closes #13619
2024-05-13 19:41:29 +02:00
Daniel Stenberg
7333faf00b
setopt: make the setstropt_userpwd args compulsory
They were always used so no point in allowing them to be optional.

follow-up to 0e37b42dc9

Closes #13608
Reviewed-by: Daniel Gustafsson
2024-05-13 10:24:01 +02:00
Daniel Gustafsson
266baf2d34 websocket: Avoid memory leak in error path
In the errorpath for randstr being too long to copy into the buffer
we leak the randstr when returning CURLE_FAILED_INIT.  Fix by using
an explicit free on randstr in the errorpath.

Closes: #13602
Reviewed-by: Daniel Stenberg <daniel@haxx.se>
2024-05-13 09:11:23 +02:00
Daniel Gustafsson
4d96873a4d hsts: Remove single-use single-line function
The hsts_entry() function contains of a single line and is only
used in a single place in the code, so move the allocation into
hsts_create instead to improve code readability. C code usually
don't use the factory abstraction for object creation, and this
small example wasn't following our usual code style.

Closes: #13604
Reviewed-by: Daniel Stenberg <daniel@haxx.se>
2024-05-13 09:07:30 +02:00
Viktor Szakats
a35bbe8977
lib: bump hash sizes to size_t
Follow-up to cc907e80a2 #13502
Cherry-picked from #13489
Closes #13601
2024-05-12 18:53:09 +02:00
Daniel Stenberg
efe93019a7
setopt: warn on Curl_set*opt() uses not using the return value
And switch the invokes that would "set" NULL to instead just plainly
free the pointer, as those were otherwise the invokes that would ignore
the return code. And possibly confuse static code analyzers.

Closes #13591
2024-05-12 17:27:51 +02:00
Jay Satiro
3a564fecab ftp: fix build for CURL_DISABLE_VERBOSE_STRINGS
This is a follow-up to b7c7dffe which changed the FTP state change
verbose debug text (aka infof) to tracing debug text (aka trc).

Prior to this change if libcurl was without DEBUGBUILD and built with
CURL_DISABLE_VERBOSE_STRINGS (ie --disable-verbose) the build would
error.

Caught by Circle CI job openssl-no-verbose.
2024-05-10 19:14:29 -04:00
Jay Satiro
798a37b25e lib: clear the easy handle's saved errno before transfer
- Clear data->state.os_errno before transfer.

- Explain the change in behavior in the CURLINFO_OS_ERRNO doc.

- Add to the CURLINFO_OS_ERRNO doc the list of libcurl network-related
  errors that may cause the errno to be saved.

data->state.os_errno is saved before libcurl returns a network-related
failure such as connection failure. It is accessible to the user via
CURLINFO_OS_ERRNO so they can get more information about the failure.

Prior to this change it wasn't cleared before transfer, so if a user
retrieved the saved errno it could be from a previous transfer. That is
because an errno is not always saved for network-related errors.

Closes https://github.com/curl/curl/pull/13574
2024-05-10 18:26:32 -04:00
Stefan Eissing
b7c7dffe35
ftp: add tracing support
- add `Curl_trc_feat_ftp` for tracing via trace config
- add macro CURL_TRC_FTP(data, fmt, ...)
- replace DEBUGF(infof()) statements in ftp.c by CURL_TRC_FTP()
- always trace FTP connection state

Closes #13580
2024-05-10 23:39:48 +02:00
Daniel Stenberg
7b5bacdd0a
http: remove redundant check
Spotted by CodeSonar

Closes #13582
2024-05-10 23:37:09 +02:00
Viktor Szakats
e0866dd054
ldap: fix unused variables (seen on OmniOS)
```
../../lib/ldap.c: In function 'ldap_do':
  ../../lib/ldap.c:380:11: error: unused variable 'ldap_ca' [-Werror=unused-variable]
    380 |     char *ldap_ca = conn->ssl_config.CAfile;
        |           ^~~~~~~
  ../../lib/ldap.c:379:9: error: unused variable 'ldap_option' [-Werror=unused-variable]
    379 |     int ldap_option;
        |         ^~~~~~~~~~~
```
Ref: https://github.com/curl/curl/actions/runs/9033564377/job/24824192730#step:3:6059

Ref: #13583
Closes #13588
2024-05-10 23:11:55 +02:00
Daniel Stenberg
0e37b42dc9
url: make parse_login_details use memdup0
Also make the user and password arguments mandatory, since all code
paths in libcurl used them anyway.

Adapted unit test case 1620 to the new rules.

Closes #13584
2024-05-10 22:53:12 +02:00
Orgad Shaneh
fa9a564942
digest: replace strcpy for empty string with simple assignment
Closes #13586
2024-05-10 22:52:36 +02:00
Daniel Stenberg
4d38cea94f
cf-socket: don't try getting local IP without socket
In cf_tcp_connect(), it might fail and not get a socket assigned to
ctx->sock but set_local_ip() is still called which would make
getsockname() get invoked with a negative file desriptor and fail.

By adding this check, set_local_ip() will now instead blank out the
fields correctly.

Spotted by CodeSonar

Closes #13577
2024-05-10 15:54:50 +02:00
Stefan Eissing
2036bebb07
quiche: trust its timeout handling
- set the idle timeout transport parameter
  in milliseconds as documented by quiche
- do not calculate the idle timeout, rely on
  quiche handling it

Closes #13581
2024-05-10 15:53:11 +02:00
Stefan Eissing
bc6e3e6049
h3/ngtcp2: improve error handling
- identify ngtcp2 and nghttp3 error codes that are fatal
- close quic connection on fatal errors
- refuse further filter operations once connection is closed
- confusion about the nghttp3 API. We should close the QUIC stream on
  cancel and not use the nghttp3 calls intended to be invoked when the
  QUIC stream was closed by the peer.

Closes #13562
2024-05-10 09:29:19 +02:00
Daniel Stenberg
0f4c439fc7
krb5: use dynbuf
Closes #13568
2024-05-09 10:59:13 +02:00
Daniel Gustafsson
60580f9f21 bufq: remove duplicate word in comment
Inspired by 13552.

Closes: #13554
Reviewed-by: Daniel Stenberg <daniel@haxx.se>
2024-05-08 19:12:03 +02:00
Viktor Szakats
bbeeccdea8
lib/cf-h1-proxy: silence compiler warnings (gcc 14)
They came up ealier with gcc 12 (Windows), but apparently gcc 14 is
still reporting them, also under Linux.

```
/home/runner/work/curl-for-win/curl-for-win/curl/lib/cf-h1-proxy.c: In function 'cf_h1_proxy_close':
/home/runner/work/curl-for-win/curl-for-win/curl/lib/cf-h1-proxy.c:1060:17: warning: null pointer dereference [-Wnull-dereference]
 1060 |   cf->connected = FALSE;
/home/runner/work/curl-for-win/curl-for-win/curl/lib/cf-h1-proxy.c:1061:8: warning: null pointer dereference [-Wnull-dereference]
 1061 |   if(cf->ctx) {
      |      ~~^~~~~
In function 'tunnel_free',
    inlined from 'cf_h1_proxy_destroy' at /home/runner/work/curl-for-win/curl-for-win/curl/lib/cf-h1-proxy.c:1053:3:
/home/runner/work/curl-for-win/curl-for-win/curl/lib/cf-h1-proxy.c:198:27: warning: null pointer dereference [-Wnull-dereference]
  198 |   struct h1_tunnel_state *ts = cf->ctx;
      |                           ^~
```
Ref: https://github.com/curl/curl-for-win/actions/runs/8985369476/job/24679219528#step:3:6320

Fixes #13237
Closes #13555
2024-05-08 12:37:33 +02:00
MAntoniak
1ea7dce08d
mbedtls: support TLS 1.3
Closes #13539
2024-05-08 11:11:45 +02:00
Daniel Stenberg
7c8970e95f
version: use msnprintf instead of strncpy
- to ensure a terminating null byte
- to avoid zero-padding the target

debug code only

Closes #13549
2024-05-08 10:40:58 +02:00
Daniel Stenberg
cd3463d99e
curl_path: make Curl_get_pathname use dynbuf
... instead of malloc and memcpy

- unit test 2604 verifies Curl_get_pathname()

Closes #13550
2024-05-08 10:10:12 +02:00
Daniel Stenberg
c294f9cb56
lib: make protocol handlers store scheme name lowercase
- saves a lowercase operation when the "[scheme]_proxy" name is
  generated
- appears less "shouting"
- update test 970, 972, 1438 and 1536

Closes #13553
2024-05-08 09:39:30 +02:00
Daniel Stenberg
848c104639
lib: remove two instances of "only only" messages
Fixes #13551
Reported-by: Lucas Nussbaum
Closes #13552
2024-05-08 00:03:38 +02:00
Pavel P
428579f5d1 asyn-thread: fix curl_global_cleanup crash in Windows
- Make sure that asynchronous resolves handled by Winsock are stopped
  before WSACleanup is called.

This is implemented by ensuring that when Curl_resolver_kill is called
(eg via multi_done) it will cancel the Winsock asynchronous resolve and
wait for the cancellation to complete. Winsock runs the asynchronous
completion routine immediately when a resolve is canceled.

Prior to this change it was possible that during curl_global_cleanup
"a DNS resolver thread created by GetAddrInfoExW did not terminate yet,
however curl is already shutting down, deinitializing Winsock with
WSACleanup() leading to an access violation."

Background:

If libcurl is built with the asynchronous threaded resolver option for
Windows then it resolves in one of two ways. For Windows 8.1 and later,
libcurl resolves by using the Winsock asynchronous resolver which does
its own thread management. For older versions of Windows, libcurl
resolves by creating a separate thread that calls getaddrinfo. This
change only affects the former and it's already handled for the latter.

Reported-by: Ch40zz@users.noreply.github.com

Fixes https://github.com/curl/curl/issues/13509
Closes https://github.com/curl/curl/pull/13518
2024-05-07 14:54:11 -04:00
Jay Satiro
62ae1f10e5 asyn-thread: fix Curl_thread_create result check
- Compare to curl_thread_t_null instead of 0 for error.

Currently for both supported thread libraries (pthreads and Windows)
curl_thread_t_null is defined as 0. However, the pattern throughout the
code is to check against curl_thread_t_null and not 0 since for
posterity some thread library may not use 0 for error.

Closes https://github.com/curl/curl/pull/13542
2024-05-07 02:28:58 -04:00
Jay Satiro
22cde3baa8 curl_multibyte: remove access() function wrapper for Windows
- Remove curlx_win32_access() which was a wrapper to use access() in
  Windows.

This is a follow-up to 602fc213, one of two commits which removed
access() calls from the codebase and banned use of the function.

Closes https://github.com/curl/curl/pull/13529
2024-05-07 02:28:10 -04:00
Ivan
a74f4d9518 misc: fix typos, quoting and spelling
Fix wording of comments, and misquotings where `' is markdown parsed
where it shouldn't be, and remove a misspelled preprocessor comment
which really isn't needed (and removing it makes it match surrounding
code better).

Closes: #13538
Reviewed-by: Daniel Gustafsson <daniel@yesql.se>
2024-05-06 10:35:53 +02:00