Commit Graph

32514 Commits

Author SHA1 Message Date
Daniel Stenberg
6ceb23dc09
cmdline-opts: expand the parallel explanations
Closes #13886
2024-06-05 10:34:55 +02:00
Daniel Stenberg
4a25812ee1
RELEASE-NOTES: synced 2024-06-05 09:09:17 +02:00
Stefan Eissing
937ba94ed5
vtls: new io_need flags for poll handling
- decouple need to recv/send from negotiation state, we need
  this later in shutdown handling as well
- move ssl enums from urldata.h to vtls_int.h
- implement use of `connssl->io_need` in vtls.c. and all backends

Closes #13879
2024-06-05 09:03:38 +02:00
Daniel Stenberg
f75aa2857f
cfilters: make Curl_conn_connect always assign 'done'
It could return error without assigning it, and we have a caller in
multi.c that assumes it gets set.

Spotted by CodeSonar
Closes #13884
2024-06-05 08:51:14 +02:00
Daniel Stenberg
1313da9a1d
CURLOPT_INTERFACE.md: quote the less-than and larger-than
Fixes the warnings shown on stderr.

Follow-up from 3060557af7

Closes #13883
2024-06-05 08:44:58 +02:00
Daniel Stenberg
68680ba544
cmdline-opts/interface.md: expand the documentation
Explain the syntax it supports.

Closes #13882
2024-06-05 08:36:34 +02:00
Daniel Stenberg
b049388d47
url: allow DoH transfers to override max connection limit
When reaching the set maximum limit of allowed connections, allow a new
connection anyway if the transfer is created for the (internal) purpose
of doing a DoH name resolve. Otherwise, unrelated "normal" transfers can
starve out new DoH requests making it impossible to name resolve for new
transfers.

Bug: https://curl.se/mail/lib-2024-06/0001.html
Reported-by: kartatz
Closes #13880
2024-06-05 08:35:12 +02:00
Viktor Szakats
998b17ea7f
windows: fix UWP builds, add GHA job
Add new job to test building for UWP (aka `CURL_WINDOWS_APP`).

Fix fallouts when building for UWP:
- rand: do not use `BCryptGenRandom()`.
- cmake: disable using win32 LDAP.
- cmake: disable telnet.
- version_win32: fix code before declaration.
- schannel: disable `HAS_MANUAL_VERIFY_API`.
- schannel: disable `SSLSUPP_PINNEDPUBKEY`
  and make `schannel_checksum()` a stub.
  Ref: e178fbd40a #1429
- schannel: make `cert_get_name_string()` a failing stub.
- system_win32: make `Curl_win32_impersonating()` a failing stub.
- system_win32: try to fix `Curl_win32_init()` (untested).
- threads: fix to use `CreateThread()`.
- src: disable searching `PATH` for the CA bundle.
- src: disable bold text support and capability detection.
- src: disable `getfiletime()`/`setfiletime()`.
- tests: make `win32_load_system_library()` a failing stub.
- tests/server/util: make it compile.
- tests/server/sockfilt: make it compile.
- tests/lib3026: fix to use `CreateThread()`.

See individual commits for build error details.

Some of these fixes may have better solutions, and some may not work
as expected. The goal of this patch is to make curl build for UWP.

Closes #13870
2024-06-05 00:52:24 +02:00
Orgad Shaneh
3060557af7
socket: support binding to interface *AND* IP
Introduce new notation for CURLOPT_INTERFACE / --interface:
ifhost!<interface>!<host>

Binding to an interface doesn't set the address, and an interface can
have multiple addresses.

When binding to an address (without interface), the kernel is free to
choose the route, and it can route through any device that can access
the target address, not necessarily the one with the chosen address.

Moreover, it is possible for different interfaces to have the same IP
address, on which case we need to provide a way to be more specific.

Factor out the parsing part of interface option, and add unit tests:
1663.

Closes #13719
2024-06-04 23:47:54 +02:00
Andy Pan
23fe1a52dc
socketpair: add eventfd and use SOCK_NONBLOCK for socketpair()
Currently, we use `pipe` for `wakeup_create`, which requires ***two***
file descriptors. Furthermore, given its complexity inside, `pipe` is a
bit heavyweight for just a simple event wait/notify mechanism.

`eventfd` would be a more suitable solution for this kind of scenario,
kernel also advocates for developers to use `eventfd` instead of `pipe`
in some simple use cases:

    Applications can use an eventfd file descriptor instead of a pipe
    (see pipe(2) in all cases where a pipe is used simply to signal
    events. The kernel overhead of an eventfd file descriptor is much
    lower than that of a pipe, and only one file descriptor is required
    (versus the two required for a pipe).

This change adds the new backend of `eventfd` for `wakeup_create` and
uses it where available, eliminating the overhead of `pipe`. Also, it
optimizes the `wakeup_create` to eliminate the system calls that make
file descriptors non-blocking by moving the logic of setting
non-blocking flags on file descriptors to `socketpair.c` and using
`SOCK_NONBLOCK` for `socketpair(2)`, `EFD_NONBLOCK` for `eventfd(2)`.

Ref:
https://man7.org/linux/man-pages/man7/pipe.7.html
https://man7.org/linux/man-pages/man2/eventfd.2.html
https://man7.org/linux/man-pages/man2/socketpair.2.html
https://www.gnu.org/software/gnulib/manual/html_node/eventfd.html

Closes #13874
2024-06-04 23:45:36 +02:00
renovate[bot]
b71916b859
ci: update github/codeql-action digest to 2e230e8
Closes #13881
2024-06-04 23:23:05 +02:00
Jay Satiro
fd2315e5ae examples/threaded-ssl: remove locking callback code
- Remove the locking callback code that demonstrates how to meet
  requirements of threading libraries (mainly OpenSSL).

Locking callback code has not been needed for many years. According to
the documentation for OpenSSL and GnuTLS they are thread-safe by design,
assuming support for the underlying OS threading API is built-in.

Ref: https://github.com/curl/curl/pull/13850#issuecomment-2143538458

Closes https://github.com/curl/curl/pull/13851
2024-06-04 13:00:21 -04:00
Viktor Szakats
75f182ffb6
tests: delete redundant !MSDOS guard
This fix was supposed to be committed earlier, but ended up missing from
the final commit.

Follow-up to e9a7d4a1c8 #12376
Closes #13878
2024-06-04 18:47:18 +02:00
Viktor Szakats
83384669ef
lib: fix thread entry point to return DWORD on WinCE
We already do this in `tests/server/util.c`:
97e5e37cc8/tests/server/util.c (L604-L606)
and in `sockfilt.c`, `lib3026.c`.

Before this patch it returned `unsigned int`.

Closes #13877
2024-06-04 18:47:18 +02:00
Andy Pan
3392f0f97e
socket: use SOCK_NONBLOCK to eliminate extra system call
Every time function `cf_socket_open()` is called to create a socket,
`curlx_nonblock()` is called to make that socket non-blocking. And
`curlx_nonblock()` will cost us 1 or 2 system calls (2 for `fcntl()`, 1
for `ioctl()`, etc.), meanwhile, tucking `SOCK_NONBLOCK` and
`SOCK_CLOEXEC` into the `type` argument for `socket()` is widely
supported across UNIX-like OS: Linux, *BSD, Solaris, etc. With that
ability, we can save 1 or 2 system calls on each socket.

Another change in this PR is to eliminate the redundant
`curlx_nonblock()` call on the socket in `cf_udp_setup_quic()` as that
socket created by `cf_socket_open()` is already non-blocking.

Ref:
https://man7.org/linux/man-pages/man2/socket.2.html
https://man.freebsd.org/cgi/man.cgi?socket(2)
https://man.dragonflybsd.org/?command=socket&section=2
https://man.netbsd.org/socket.2
https://man.openbsd.org/socket
https://docs.oracle.com/cd/E88353_01/html/E37843/socket-3c.html
https://illumos.org/man/3SOCKET/socket
...

Closes #13855
2024-06-04 17:51:28 +02:00
Viktor Szakats
97e5e37cc8
GHA: show cmake error log in Windows and non-native workflows
CMake configure doesn't fail often, but when it does, it helps to see
its `CMakeFiles/CMakeConfigureLog.yaml` output. This file is present
since CMake v3.26:
  https://cmake.org/cmake/help/v3.26/manual/cmake-configure-log.7.html

(Older CMake versions save similar contend to
`CMakeFiles\CMakeOutput.log` and
`CMakeFiles\CMakeError.log`. This patch doesn't deal with that because
the workflows touched are all running a newer CMake.)

After this patch, we dump the content if cmake fails. Syncing this with
autotools, where we already did that.

Closes #13872
2024-06-04 11:54:32 +02:00
Viktor Szakats
23640dcf12
GHA: switch a Windows job to UCRT (gcc)
Cherry-picked from #13870
2024-06-04 11:20:23 +02:00
Viktor Szakats
dada6736df
curl-config: revert to backticks to support old target envs
Make an exception for `curl-config` because this script that may be
running on any target system, including old ones, e.g. SunOS 5.10.

Reported-by: Alejandro R. Sedeño
Ref: https://github.com/curl/curl/pull/13307#issuecomment-2146427358
Follow-up to fa69b41c77 #13307
Closes #13871
2024-06-04 10:13:21 +02:00
Stefan Eissing
5f9017d4e2
mbedtls: v3.6.0 workarounds
- add special sauce to disable unwanted peer verification by mbedtls
  when negotiating TLS v1.3
- add special sauce for MBEDTLS_ERR_SSL_RECEIVED_NEW_SESSION_TICKET
  return code on *writing* TLS data. We assume the data had not been
  written and EAGAIN.
- return correct Curl error code when peer verification failed.
- disable test_08_05 with 50 HTTP/1.1 connections, as mbedtls reports a
  memory allocation failed during handshake.
- bump CI mbedtls version to 3.6.0

Fixes #13653
Closes #13838
2024-06-04 09:02:37 +02:00
Stefan Eissing
5dd8f13bfc
gnutls: support CA caching
- similar to openssl, use a shared 'credentials' instance
  among TLS connections with a plain configuration.
- different to openssl, a connection with a client certificate
  is not eligible to sharing.
- document CURLOPT_CA_CACHE_TIMEOUT in man page

Closes #13795
2024-06-04 08:17:55 +02:00
Dan Fandrich
4a7bb40dea tests: don't log buffer length in throwing away message
It's not available at that point, and it will be written in the
non-error case right afterward.
2024-06-03 22:21:09 -07:00
Dan Fandrich
bc21c505e4 tests: log "Throwing away" messages before throwing away
In case the read that follows hangs we'll get a clue as to what it was
doing.
2024-06-03 17:00:42 -07:00
Dan Fandrich
5b35dea6ef CI: reduce memory request for FreeBSD builds
Also, add a comment with link to the Cirrus credit page since it's not
easy to find otherwise.
2024-06-03 17:00:42 -07:00
Andy Pan
f51fa8f169
tcpkeepalive: support setting TCP keep-alive parameters on Solaris <11.4
Solaris didn't support TCP_KEEPIDLE and TCP_KEEPINTVL until 11.4,
before that it use TCP_KEEPALIVE_THRESHOLD and TCP_KEEPALIVE_ABORT_THRESHOLD
as the substitute. Therefore, for Solaris <11.4 we need to use this substitute
for setting TCP keep-alive parameters.

Ref:
https://docs.oracle.com/cd/E86824_01/html/E54777/tcp-7p.html
https://docs.oracle.com/cd/E88353_01/html/E37851/tcp-4p.html

Closes #13864
2024-06-03 23:04:05 +02:00
Daniel Stenberg
4edbd52267
KNOWN_BUGS: quiche: QUIC connection is draining
Closes #12037
Closes #13867
2024-06-03 10:46:10 +02:00
Daniel Stenberg
8e5c5c1724
KNOWN_BUGS: aws-sigv4 has problems with particular URLs
Closes #13058
Closes #13866
2024-06-03 10:45:24 +02:00
Daniel Stenberg
2a053ca801
KNOWN_BUGS: aws-sigv4 does not handle multipart/form-data correctly
Closes #13351
Closes #13866
2024-06-03 10:45:16 +02:00
Daniel Stenberg
6c31c65a8c
RELEASE-NOTES: synced 2024-06-03 10:43:06 +02:00
Viktor Szakats
fd149601cc
GHA: fix old mingw-w64 32-bit job
This toolchain resides in the `mingw32` directory. Make sure to
configure `PATH` accordingly.

Before this patch, it pointed to a non-existing `mingw64` directory,
making the job use the wrong compiler (gcc 12, 64-bit).

Follow-up to e838b341a0 #12927
Closes #13863
2024-06-03 01:34:04 +02:00
Daniel Stenberg
c61f753881
tool_cb_hdr: return error for failed header writes
By checking that fflush() works.

Reported-by: Sebastian Andersson
Fixes #13836
Closes #13859
2024-06-02 23:24:19 +02:00
Viktor Szakats
464282ddfb
GHA: bump all build jobs to nproc+1
- bump rest of the workflows (windows, macos, distrocheck).

- non-native virtualized envs have 2 CPUs, bump down accordingly.
  (for `vmactions/omnios-vm` it's just a guess.)

- bump all to nproc + 1.

Follow-up to e838b341a0 #12927
Closes #13807
2024-06-02 21:39:49 +02:00
Viktor Szakats
03bd16e533
GHA: disable MQTT and WebSocket tests in Windows jobs
Trying to figure out which category is causing the remaining hangs.

Follow-up to def7d05382
Closes #13860
2024-06-02 20:29:46 +02:00
Viktor Szakats
0887297100
lib/v*: tidy up types and casts
Also add a couple of negative checks.

Cherry-picked from #13489
Closes #13622
2024-06-02 19:27:17 +02:00
Viktor Szakats
1d63e331de
GHA: fix caching old mingw-w64 toolchains in the Windows workflow
- stop altering the `PATH` via `GITHUB_ENV`. This confused the
  `actions/cache` post-job, which needs to run in the exact same
  environment as its pre-job, to have a consistent cache entry "version"
  hash. Altering the `PATH` via `GITHUB_ENV` spills into the the
  post-job and breaks this hash. GHA doesn't reset the env automatically
  and I have not found a way to do it manually.

- add double-quotes where missing.

- move cache directory under `USERPROFILE` to not rely on absolute
  paths.

- make cache directory flatter and versionless.

Follow-up to 0914d8aadd #13759
Closes #13856
2024-06-02 19:27:16 +02:00
renovate[bot]
7d7712166f
ci: pin actions/github-script action to 60a0d83
Closes #13846
2024-06-02 17:34:45 +02:00
Bo Anderson
eacec9a350
x509asn1: add some common ECDSA OIDs
Closes #13857
2024-06-02 17:30:09 +02:00
renovate[bot]
aa954f97f6
ci: update rojopolis/spellcheck-github-actions digest to e36f662
Closes #13852
2024-06-02 17:25:37 +02:00
Bo Anderson
9aa1d412b8
x509asn1: fallback to dotted OID representation
Reported-by: Luke Hamburg
Fixes #13845
Closes #13858
2024-06-02 17:23:50 +02:00
Lee Li
6a7ad8911d
request.md: language fix
improved for better readability and correctness

Closes #13854
2024-06-02 17:12:25 +02:00
MonkeybreadSoftware
810933d779
vtls: deprioritize Secure Transport
Moved Secure Transport behind OpenSSL, so we can build CURL with both
and prefer using OpenSSL over Secure Transport by default.

Closes #13547
2024-06-02 17:10:17 +02:00
Daniel Stenberg
655d44d139
urlapi: add CURLU_NO_GUESS_SCHEME
Used for extracting:

- when used asking for a scheme, it will return CURLUE_NO_SCHEME if the
  stored information was a guess

- when used asking for a URL, the URL is returned without a scheme, like
  when previously given to the URL parser when it was asked to guess

- as soon as the scheme is set explicitly, it is no longer internally
  marked as guessed

The idea being:

1. allow a user to figure out if a URL's scheme was set as a result of
  guessing

2. extract the URL without a guessed scheme

3. this makes it work similar to how we already deal with port numbers

Extend test 1560 to verify.

Closes #13616
2024-06-01 23:51:42 +02:00
Daniel Stenberg
80aa519545
wolfssl: support CA caching
As a bonus, add SSLSUPP_CA_CACHE to let TLS backends signal its support
for this so that *setopt() return error if there is no support.

Closes #13786
2024-06-01 23:50:36 +02:00
Andy Pan
6da320357f
socket: change TCP keepalive from ms to seconds on DragonFly BSD
DragonFly BSD changed the time unit for TCP keep-alive from milliseconds
to seconds since v5.8, thus setting the keepalive options with
milliseconds with curl/libcurl will result in unexpected behaviors on
DragonFlyBSD 5.8+

Distinguish the DragonFly BSD versions and use the proper time units
accordingly.

Ref:
  https://lists.dragonflybsd.org/pipermail/commits/2019-July/719125.html
  965b380e96/sys/sys/param.h (L207)

Fixes #13847
Closes #13848
2024-06-01 23:46:41 +02:00
Daniel Stenberg
47d930efae
curlver.h: aiming for 8.9.0 2024-06-01 12:26:20 +02:00
Daniel Stenberg
e78913e6b5
noproxy: patterns need to be comma separated
or they will not parse correctly.

Mentioned in DEPRECATED since Janurary 2023 (in 7ad8a7ba9e).

Closes #13789
2024-06-01 12:25:13 +02:00
Jan Venekamp
4e2c45110c
sectransp: remove large cipher table
Previously a large table of ciphers was used to determine the default
ciphers and to lookup manually selected ciphers names.

With the lookup of the manually selected cipher names moved to
Curl_cipher_suite_walk_str() the large table is no longer needed for
that purpose.

The list of manually selected cipher can now be intersected with the
ciphers supported by Secure Transport (SSLGetSupportedCiphers()),
instead of using the fixed table for that.

The other use of the table was to filter the list of all supported
ciphers offered by Secure Transport to create a list of ciphers to
use by default, excluding ciphers in the table marked as weak.

Instead of using a complement based approach (exclude weak), switch
to using an intersection with a smaller list of ciphers deemed
appropriate.

Closes #13823
2024-06-01 11:00:06 +02:00
Tatsuhiro Tsujikawa
a8174176b5
GHA: unify http3 workflows into one
This commit unifies the following http3 workflows into http3-linux.yml:

- ngtcp2-linux.yml
- osslq-linux.yml
- quiche-linux.yml

The idea is better use of the build cache. Previously, they
independently create caches with the same key. Some of the caches
include source code and intermediate object files, which makes cache
quite large. In this commit, only built artifacts are cached, which
drastically reduces the cache size. OpenSSL v3, mod_h2 and quiche caches
still include all stuff, but they are left for the later improvement.
Because the contents of the cache have been changed, the cache keys are
also changed to include the word "http3".

Closes #13841
2024-06-01 10:57:23 +02:00
Stephen Farrell
48292d8c93
openSSL: fix hostname handling when using ECH
Reported-by: vvb2060
Fixes #13818
Closes #13822
2024-06-01 10:48:23 +02:00
renovate[bot]
fee2e40ef6
ci: update github/codeql-action digest to f079b84
Closes #13837
2024-06-01 10:45:33 +02:00
Daniel Stenberg
361f4c65fb
RELEASE-NOTES: synced 2024-06-01 10:40:20 +02:00