Commit Graph

32849 Commits

Author SHA1 Message Date
Viktor Szakats
ea3dfcb361
cmake: drop unused HAVE_IDNA_STRERROR
Unused since 9c91ec7781

Closes #14420
2024-08-06 22:33:05 +02:00
Viktor Szakats
6712bd6006
cmake: show CMake platform/compiler flags
To better understand which is active in which build job.

Closes #14417
2024-08-06 22:33:04 +02:00
Daniel Stenberg
dcc5209520
GHA: run badwords check on tests/*.md too
With several wording updates accordingly.

Closes #14412
2024-08-06 13:43:25 +02:00
Daniel Stenberg
91fcbc5d1a
dist: drop buildconf
The documented and mandated step has been to not use buildconf but to
invoke 'autoreconf -fi' for four years already.

This change only drops buildconf from the release tarball, it remains
present in git for now.

Follow-up to 85868537d6

Closes #14412
2024-08-06 13:43:20 +02:00
Tal Regev
8577f4ca08
cmake: add math library when using wolfssl and ngtcp2
Closes #14343
2024-08-06 11:48:21 +02:00
Yedaya Katsman
bfa939d062
docs: mention "@-" in more places
Closes #14402
2024-08-06 11:47:27 +02:00
Viktor Szakats
67d5e3624b
cmake: replace an MSVC_VERSION with MSVC
Where the actual version is not relevant.

Follow-up to ce81aeb877
Closes #14410
2024-08-06 11:05:49 +02:00
Viktor Szakats
72ae0d86a4
cmake: use numeric comparison for HAVE_WIN32_WINNT
Turns out CMake supports numeric comparison with hexadecimal values.
Confirmed in GHA/linux-old with CMake 3.7.2. I could not find
documentation about this, but our CMakeLists.txt already used it before
this patch.

Extend that method to two more comparisons.

Also pad the value in the existing one to 4 digits.

The padding/lowercasing logic when setting `HAVE_WIN32_WINNT` is no
longer required, but keep it anyway for uniform log output.

Follow-up to 2100d9fde2 #12044
Closes #14409
2024-08-06 11:05:48 +02:00
Daniel Stenberg
8de8fe8c98
configure: detect AppleIDN
Follow-up to add22feeef

Closes #14401
2024-08-06 09:36:07 +02:00
Viktor Szakats
232302f88a
cmake: add Linux CI job, fix pytest with cmake
- extend existing Linux workflow with CMake support.
  Including running pytest the first time with CMake.

- cmake: generate `tests/config` and `tests/http/config.ini`.
  Required for pytest tests.
  Uses basic detection logic. Feel free to take it from here.
  Also dump config files in a CI step for debugging purposes.

- cmake: build `tests/http/clients` programs.

- fix portability issues with `tests/http/clients` programs.
  Some of them use `getopt()`, which is not supported by MSVC.
  Fix the rest to compile in CI (old-mingw-w64, MSVC, Windows).

- GHA/linux: add CMake job matching an existing autotools one.

- GHA/linux: test `-DCURL_LIBCURL_VERSIONED_SYMBOLS=ON`
  in the new CMake job.

- reorder testdeps to build server, client tests first and then
  libtests and units, to catch errors in the more complex/unique
  sources earlier.

- sort list in `tests/http/clients/Makefile.inc`.

Closes #14382
2024-08-06 02:40:32 +02:00
Viktor Szakats
f7d5f47059
cmake: add support for CURL_USE_LIBUV option
Also use an `#undef` hack for CMake Unity builds to avoid the previously
included `memdebug.h` header messing up the declarations pulled in by
`uv.h`:
```
In file included from ~/curl/bld/src/CMakeFiles/curl.dir/Unity/unity_0_c.c:88:
In file included from ~/curl/src/tool_operate.c:54:
In file included from /usr/local/Cellar/libuv/1.48.0/include/uv.h:71:
In file included from /usr/local/Cellar/libuv/1.48.0/include/uv/unix.h:34:
/Library/Developer/CommandLineTools/SDKs/MacOSX13.1.sdk/usr/include/netdb.h:269:7: error: expected parameter declarator
void            freeaddrinfo(struct addrinfo *);
                ^
~/curl/lib/memdebug.h:167:31: note: expanded from macro 'freeaddrinfo'
  curl_dbg_freeaddrinfo(data, __LINE__, __FILE__)
                              ^
```

Follow-up to 38d334e3e1 #14298
Closes #14399
2024-08-06 02:40:16 +02:00
dependabot[bot]
e64e62cc7d
GHA/windows: bump msys2/setup-msys2 from 2.24.0 to 2.24.1
Bumps [msys2/setup-msys2](https://github.com/msys2/setup-msys2) from 2.24.0 to 2.24.1.
- [Release notes](https://github.com/msys2/setup-msys2/releases)
- [Changelog](https://github.com/msys2/setup-msys2/blob/main/CHANGELOG.md)
- [Commits](5df0ca6cbf...ddf331adae)

---
updated-dependencies:
- dependency-name: msys2/setup-msys2
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <support@github.com>
Closes #14396
2024-08-05 23:35:18 +02:00
Austin Moore
cf3e3d93d1
aws_sigv4: fix canon order for headers with same prefix
If a request containing two headers that have equivalent prefixes (ex.
"x-amz-meta-test:test" and "x-amz-meta-test-two:test2") AWS expects the
header with the shorter name to come first. The previous implementation
used `strcmp` on the full header. Using the example, this would result
in a comparison between the ':' and '-' chars and sort
"x-amz-meta-test-two" before "x-amz-meta-test", which produces a
different "StringToSign" than the one calculated by AWS.

Test 1976 verifies

Closes #14370
2024-08-05 23:32:10 +02:00
Jan Venekamp
f3e07e5c55
docs: wolfssl and mbedtls add CURLOPT_TLS13_CIPHERS support
Documentation for #14384 and #14385

Closes #14386
2024-08-05 23:29:11 +02:00
Jan Venekamp
4c12892411
wolfssl: add CURLOPT_TLS13_CIPHERS support
Bring setting ciphers with WolfSSL in line with other SSL backends,
to make the curl interface more consistent across the backends.

Now the tls1.3 ciphers are set with the --tls13-ciphers option, when
not set the default tls1.3 ciphers are used. The tls1.2 (1.1, 1.0)
ciphers are set with the --ciphers option, when not set the default
tls1.2 ciphers are used. The ciphers available for the connection
are now a union of the tls1.3 and tls1.2 ciphers.

This changes the behaviour for WolfSSL when --ciphers is set, but
--tls13-ciphers is not set. Now the ciphers set with --ciphers
are combined with the default tls1.3 ciphers, whereas before solely
the ciphers of --ciphers were used.

Thus before when no tls1.3 ciphers were specified in --ciphers,
tls1.3 was completely disabled. This might not be what the user
expected, especially as this does not happen with OpenSSL.

Closes #14385
2024-08-05 23:28:04 +02:00
Daniel Stenberg
a18680f501
VULN-DISCLOSURE-POLICY.md: small typo fix 2024-08-05 17:15:31 +02:00
Viktor Szakats
82bbb386a3
cmake: fix GSS_VERSION for Heimdal found via pkg-config
Previously used source variable was never defined, possibly due
to a copy-paste-edit typo.

Closes #14393
2024-08-05 16:37:19 +02:00
Jan Venekamp
3f7dc8a404
mbedtls: add CURLOPT_TLS13_CIPHERS support
Bring setting ciphers with mbedTLS in line with other SSL backends,
to make the curl interface more consistent across the backends.

Now the tls1.3 ciphers are set with the --tls13-ciphers option, when
not set the default tls1.3 ciphers are used. The tls1.2 (1.1, 1.0)
ciphers are set with the --ciphers option, when not set the default
tls1.2 ciphers are used. The ciphers available for the connection
are now a union of the tls1.3 and tls1.2 ciphers.

This changes the behaviour for mbedTLS when --ciphers is set, but
--tls13-ciphers is not set. Now the ciphers set with --ciphers
are combined with the default tls1.3 ciphers, whereas before solely
the ciphers of --ciphers were used.

Thus before when no tls1.3 ciphers were specified in --ciphers,
tls1.3 was completely disabled. This might not be what the user
expected, especially as this does not happen with OpenSSL.

Closes #14384
2024-08-05 16:01:20 +02:00
Moritz Buhl
d266d19d86
ngtcp2: use NGHTTP3 prefix instead of NGTCP2 for errors in h3 callbacks
Closes #14394
2024-08-05 15:58:45 +02:00
Daniel Stenberg
b9d465c89f
tool_help: fix a NULL deref in the --help option code
Follow-up to 9a0cf56471

Pointed out by CodeSonar

Closes #14391
2024-08-05 15:55:38 +02:00
Daniel Stenberg
0238a9b0d7
KNOWN_BUGS: "special characers" in URL works with aws-sigv4
Since #13754
Pointed-out-by: Slaven Rezić
Bug: https://github.com/curl/curl/issues/13754#issuecomment-2268374427
Closes #14392
2024-08-05 15:25:47 +02:00
Daniel Stenberg
38d334e3e1
curl: use libuv for parallel transfers with --test-event
add --with-libuv to configure to (optionally) use it in debug-builds to
drive the event-based API

Use curl_multi_socket_action() and friends to drive parallel transfers.

tests/README has brief documentation for this

Closes #14298
2024-08-05 09:12:09 +02:00
Daniel Stenberg
7c31ceb5d0
RELEASE-NOTES: synced 2024-08-05 09:03:11 +02:00
Stefan Eissing
35bf766280
http2: improved upload eos handling
- replace the counting of upload lengths with the new eos send flag
- improve frequency of stream draining to happen less on events where it
  is not needed
- this PR is based on #14220

http2, cf-h2-proxy: fix EAGAINed out buffer
- in adjust pollset and shutdown handling, a non-empty `ctx->outbufq`
  must trigger send polling, irregardless of http/2 flow control
- in http2, fix retry handling of blocked GOAWAY frame

test case improvement:
- let client 'upload-pausing' handle http versions

Closes #14253
2024-08-05 08:53:55 +02:00
Stefan Eissing
344ba8c883
wolfssl: improve shutdown handling
Improve handling of shutdown when sending gets blocked.

Add workaround for <https://github.com/wolfSSL/wolfssl/issues/7784>
where wolfSSL keeps on adding close notify messages to its outgoing
buffer on ever attempt.

Closes #14376
2024-08-05 08:49:20 +02:00
Stefan Eissing
4494005b50
openssl: improve shutdown handling
Make sure that `io_need` is cleared and set at the filter operations.
Add some more tracing for shutdown situations.

Improve shutdown handling for blocked sends. OpenSSL is a bit tricksy
here that it only reports WANT_WRITE on SSL_shutdown(), but never on
SSL_read() on blocked sends. So we need to use both.

At last, set SSL_MODE_ACCEPT_MOVING_WRITE_BUFFER when available since we
are not always retrying sends from the very same address, as testing
showed.

Closes #14375
2024-08-05 08:48:06 +02:00
Stefan Eissing
6f19210667
bearssl: improve shutdown handling
- replace own "adjust_pollset()" with vtls' standard one
- set io_need always accordingly
- handle EAGAIN on shutdown

Closes #14374
2024-08-05 08:46:24 +02:00
Daniel Stenberg
ed2850456c
configure: fail if PSL is not disabled but not found
Regression since 9b3f67e (shipped in 8.7.0)
Reported-by: Ryan Carsten Schmidt
Fixes #14373
Assisted-by: Viktor Szakats
Closes #14379
2024-08-05 08:33:58 +02:00
Daniel Stenberg
7d45b52166
KNOWN_BUGS: mention AppleIDN and WinIDN test problems
Closes #14176
Closes #14387
2024-08-05 08:23:37 +02:00
Tal Regev
781c14c4ed
tool_operhlp: fix "potentially uninitialized local variable 'pc' used"
A false positive by MSVC.

Closes #14389
2024-08-05 08:21:49 +02:00
Daniel Stenberg
3eec5afbd0
sigpipe: init the struct so that first apply ignores
Initializes 'no_signal' to TRUE, so that a call to sigpipe_apply() after
init ignores the signal (unless CURLOPT_NOSIGNAL) is set.

I have read the existing code multiple times now and I think it gets the
initial state reversed this missing to ignore.

Regression from 17e6f06ea3

Reported-by: Rasmus Thomsen
Fixes #14344
Closes #14390
2024-08-05 08:18:43 +02:00
Alex Snast
8d98118026
wolfssl: add support for ssl cert blob / ssl key blob options
wolfSSL supports setting certificates/private keys from memory blobs
which allow us to implement both CURLOPT_SSLCERT_BLOB and
CURLOPT_SSLKEY_BLOB options.

Closes #14018
2024-08-05 08:17:30 +02:00
Viktor Szakats
7b14449790
cmake: add support for versioned symbols option
Implement the `--enable-versioned-symbols` feature available in
`./configure` for CMake.

Enable with `-DCURL_LIBCURL_VERSIONED_SYMBOLS=ON`. Customize the version
prefix with `-DCURL_LIBCURL_VERSIONED_SYMBOLS_PREFIX=MYPREFIX_`. By
default the prefix matches what autotools uses.

When enabled, the feature is detected and a warning shown if
unavailable. (E.g. on Apple and Windows, it isn't.)

Included `HIDDEN {};` to match autotools, though I don't know if it's
necessary, useful or making any difference.

Differences from the autotools implementation:
- soversion is dynamic instead of hard-coded.
- omits referencing non-curl symbols.
- allows prefix/flavour override.
- more universal feature detection.
- doesn't rely on the in-repo `lib/libcurl.vers.in` file.

Also:
- add mbedTLS and BearSSL versioned symbol prefix support to autotools.
- enable this option in an old-linux job.

Follow-up to 7cc2e8b349
Fixes #14349
Closes #14378
2024-08-05 01:44:57 +02:00
Jay Satiro
573aaec3be easy: fix curl_easy_upkeep for shared connection caches
- Determine which connection cache is or will be used by the easy handle
  and perform connection upkeep on that cache.

This change allows curl_easy_upkeep to be effective on easy handles that
are using a shared connection cache, either from a user created shared
connection cache or a user created multi which has its own shared
connection cache.

Prior to this change curl_easy_upkeep would upkeep the connection cache
for the easy handle only if that cache was from the multi owned by the
easy handle (ie curl_easy_perform was previously called and there's a
connection cache exclusive to the easy handle in
data->multi_easy->conn_cache).

Ref: https://curl.se/mail/lib-2024-01/0016.html

Closes https://github.com/curl/curl/pull/12677
2024-08-04 18:41:18 -04:00
Jay Satiro
b7e769dc87 vtls: stop offering alpn http/1.1 for http2-prior-knowledge
- For HTTPS if http2-prior-knowledge is set then only offer h2 (HTTP/2)
  alpn to the server for protocol negotiation.

Prior to this change both HTTP/2 ("h2") and HTTP/1.1 ("http/1.1") were
offered for ALPN when http2-prior-knowledge was set.

CURL_HTTP_VERSION_2_PRIOR_KNOWLEDGE (tool: --http2-prior-knowledge) is
meant to send non-TLS HTTP requests HTTP/2 when it is known the server
supports them. However when HTTPS is used then it attempts to first
negotiate the connection with ALPN. In that case the user likely does
not want to offer http/1.1 to the server as an acceptable protocol.

Reported-by: kit-ty-kate@users.noreply.github.com

Fixes https://github.com/curl/curl/issues/9963
Closes https://github.com/curl/curl/pull/14266
2024-08-04 17:59:04 -04:00
Daniel Stenberg
732cb15b97
curl: add --skip-existing
With this option, the entire download is skipped if the selected target
filename already exists when the opertion is about to begin.

Test 994, 995 and 996 verify.

Ref: #11012
Closes #13993
2024-08-04 23:28:09 +02:00
Jan Venekamp
eec908bb6e
revert "tests/http: configure test httpd to honor client cipher order"
revert f6cb3c63 #14338

Setting SSLHonorCipherOrder to on means it honors the server cipher
order. From the documentation: "When choosing a cipher during an SSLv3
or TLSv1 handshake, normally the client's preference is used. If this
directive is enabled, the server's preference will be used instead."

Also the commit inhibits test_17_07_ssl_ciphers. The test tries to
tests if all the ciphers specified, and only those, are properly set
in curl. For that to work we need have cases where some or all ciphers
do no intersect with the cipher-set of the server. We need to be able
to assert a failed connection based on a cipher set mismatch.

That is why a restricted set of ciphers is used on the server. This
set is so chosen that it contains the well known most secure ciphers.
Except with the slower aes256 variant intentionally left out, to be
able to test above described.

As test_17_07_ssl_ciphers is currently the only test that tests the
functioning of the --ciphers and --tls13-ciphers options, it is
important that its coverage is as good as possible.

Closes #14381
2024-08-04 23:03:46 +02:00
Tal Regev
8a95678999
GHA/windows: add mbedTLS MSVC job
Ref: 92e28f2897 #14228
Closes #14203
2024-08-04 19:51:45 +02:00
Viktor Szakats
f81f351b9a
tidy-up: OS names
Use these words and casing more consistently across text, comments and
one curl tool output:
AIX, ALPN, ANSI, BSD, Cygwin, Darwin, FreeBSD, GitHub, HP-UX, Linux,
macOS, MS-DOS, MSYS, MinGW, NTLM, POSIX, Solaris, UNIX, Unix, Unicode,
WINE, WebDAV, Win32, winbind, WinIDN, Windows, Windows CE, Winsock.

Mostly OS names and a few more.

Also a couple of other minor text fixups.

Closes #14360
2024-08-04 19:17:45 +02:00
Viktor Szakats
a4ad7dc5a3
dist: add missing docs/examples/CMakeLists.txt
Follow-up to dfdd978f7c #13491
Closes #14380
2024-08-04 19:12:50 +02:00
Daniel Stenberg
1159dc359d
RELEASE-NOTES: synced 2024-08-04 16:26:21 +02:00
Daniel Stenberg
0a94578a9c
maketgz: accept option to include latest commit hash
If the second argument to the script is "commit", then this will
generate a file named `docs/tarball-commit.txt` that contains the latest
commit hash (git rev-parse HEAD) at the time the script runs.

Doing this breaks the reproducibility so it will not be used for "real"
releases but is meant for automated daily snapshots and similar.

Reported-by: Dan Fandrich
Fixes #14363
Closes #14369
2024-08-04 16:21:42 +02:00
Daniel Stenberg
9a0cf56471
curl: --help [option] displays documentation for given cmdline option
Since the documentation text blob might be gzipped, it needs to search
for what to output in a streaming manner. It then first searches for
"\nALL OPTIONS".

Then, it looks for the start to display at "\n    -[option]" and stops
again at "\n    -". Except for the last option in the man page, which
ends at "\nFILES" - the subtitle for the section following all options
in the manpage.

Test 1707 to 1710 verify

Closes #13997
2024-08-04 16:06:17 +02:00
Daniel Stenberg
9b1e4b4632
tool_operate: support --dump-header % to direct to stderr
Similar to how --trace and --trace-ascii already do it.

Added test 1489 to verify

Closes #13992
2024-08-04 16:02:24 +02:00
Daniel Stenberg
e26eefd9ce
tool_operate: for -O, use "default" as filename when the URL has none
... or pick the last directory part from the path if available.

Instead of returning error.

Add test 690 and 691 to verify. Test 76 and 2036 no longer apply.

Closes #13988
2024-08-04 15:59:51 +02:00
David Sardari
cb829f994d
doh-url.md: point out DOH server IP pinning
Closes #14377
Signed-off-by: David Sardari <d@duxsco.de>
2024-08-04 15:57:45 +02:00
Viktor Szakats
4f198c8525
tests: fixup tests/data/Makefile.am references
Follow-up to f5b826532f #14357
Reported-by: Stefan Eissing
Fixes #14371
Closes #14372
2024-08-04 15:28:29 +02:00
Viktor Szakats
1556951c4b
GHA/non-native: ignore FreeBSD FTP test results
They are flaky.
2024-08-04 02:08:27 +02:00
Stefan Eissing
93d1af4011
pytests: add tests for HEAD requests in all HTTP versions
Closes #14367
2024-08-04 00:20:14 +02:00
Viktor Szakats
acbc6b703f
cmake: tidy-ups
- tidy-up comments.
- use lowercase, underscore prefixed names for internal variables.
- use `IN LISTS` and `IN ITEMS` in `foreach()` loops.
- rename variable name `OUTPUT` to a more distinctive one.
- tidy-up `STREQUAL` syntax.
- delete commented code.
- indent/whitespace.

Closes #14197
2024-08-03 20:49:15 +02:00