Commit Graph

33130 Commits

Author SHA1 Message Date
Viktor Szakats
8e9056f8b1
GHA/macos: enable brotli and zstd in autotools and cmake jobs
They were missing from macOS builds:
https://testclutch.curl.se/static/reports/feature-matrix.html

Closes #14619
2024-08-21 07:11:34 +02:00
Viktor Szakats
2e88ef1049
version: fix shadowing a libssh.h symbol
```
/Users/runner/work/curl/curl/lib/version.c: In function 'curl_version_info':
/Users/runner/work/curl/curl/lib/version.c:584:15: error: declaration of 'ssh_buffer' shadows a global declaration [-Werror=shadow]
  584 |   static char ssh_buffer[80];
      |               ^~~~~~~~~~
In file included from /Users/runner/work/curl/curl/lib/vssh/ssh.h:35,
                 from /Users/runner/work/curl/curl/lib/urldata.h:185,
                 from /Users/runner/work/curl/curl/lib/altsvc.c:32,
                 from /Users/runner/work/curl/curl/bld/lib/CMakeFiles/libcurl_shared.dir/Unity/unity_0_c.c:4:
/opt/homebrew/include/libssh/libssh.h:99:35: note: shadowed declaration is here
   99 | typedef struct ssh_buffer_struct* ssh_buffer;
      |                                   ^~~~~~~~~~
```
Ref: https://github.com/curl/curl/actions/runs/10477958747/job/29020250670#step:9:48

Tested via #14614

Closes #14617
2024-08-20 23:22:26 +02:00
Viktor Szakats
ac207bf567
ssh: deduplicate SSH backend includes (and fix libssh cmake unity build)
For libssh, it fixes a "unity" build issue where libssh deprecation
warnings were not suppressed before this patch, because the suppression
macro was only set before just one of the two `libssh.h` includes.
If the other was compiled first in unity mode, the warnings appeared.

Seen in local curl-for-win build (`CW_CONFIG=test-x64-libssh-quictls`)
with libssh 0.11.0. (Also in a GHA/macos cmake job upcoming in #14614)

Use this opportunity to drop duplicate SSH header includes from the SSH
modules. It's enough to include them via the common `ssh.h` header.

Closes #14612
2024-08-20 21:34:24 +02:00
Viktor Szakats
440d00d17e
tidy-up: spelling 'built-in'
Closes #14613
2024-08-20 21:32:40 +02:00
Dan Fandrich
e83c83807d build: improve compiler version detection portability
POSIX sed doesn't support extended regular expressions, so convert a
call to the basic format. This caused a problem on AIX. Also, use the
detected sed binary name instead of hard-coding one.
2024-08-20 11:02:31 -07:00
Viktor Szakats
ae2c753a88
GHA/windows: add missing time limit for msys2 autotools test runs
The new mingw job (the first msys2/mingw job to run tests) is seen
hanging frequently.

Follow-up to 9f23c8f201 #14541
2024-08-20 16:02:15 +02:00
Jan Venekamp
0cbfce802d
tests: add test_17_09_ssl_min_max
Test setting all combinations of --tlsv1.x and --tls-max.

Closes #14590
2024-08-20 12:48:36 +02:00
Jan Venekamp
3ca38f9a5e
tests: improve test_17_07_ssl_ciphers
Change TLS proto version on the test httpd server to test setting
combinations of --tls13-ciphers and --ciphers.

To not let the changed config of the httpd server bleed into the next
test, clean and reload on each test. Because a reload is slow, only
do this if the config is different than the loaded config. For this
the httpd.reload_if_config_changed() method is added.

Overloading of autouse fixtures does not seem to work. For the test
httpd server to be reloaded with a clean config in test_18_methods,
to not be affected by the config changes in test_17_ssl_use, the two
class scope fixtures of test_18_methods are now combined.

Closes #14589
2024-08-20 12:46:13 +02:00
Jan Venekamp
925aea1aba
mbedtls: no longer use MBEDTLS_SSL_VERIFY_OPTIONAL
With mbedTLS if the minimum version of TLS is set to 1.3,
MBEDTLS_SSL_VERIFY_OPTIONAL is not available in client mode. See:
https://github.com/Mbed-TLS/mbedtls/blob/2ca6c285/library/ssl_tls.c#L1357
Also, there might be plans to remove it completely in future mbedTLS
versions.

Switch to always use MBEDTLS_SSL_VERIFY_REQUIRED. If verifypeer or
verifyhost are disabled the corresponding error flags are cleared in the
verify callback function. That is also where verification errors are
logged.

Closes #14591
2024-08-20 12:45:06 +02:00
renovate[bot]
e8bfa96393
GHA: update github/codeql-action digest to 883d858
Closes #14608
2024-08-20 12:37:37 +02:00
Viktor Szakats
422696f0a4
cmake: migrate dependency detections to Find modules
For: libgsasl, libidn2, libssh, libuv.

The new Find modules retain using `pkg-config` natively, not as a "hint"
for the CMake-native detection. Of the pre-existing Find modules, only
FindNettle, and FindGSS (with customized code) work this way. Align
detection code for the new modules and add version detection for the
CMake-native paths.

Also, add CMake-native detection for `libgsasl`.

The remaining outlier in `CMakeLists.txt` is GnuTLS, which has
a CMake built-in Find module, but which lacks `pkg-config` support,
required for vcpkg. It remains unchanged.

Another part-outlier is `libssh`, which keeps requiring the trick
`find_package(libssh CONFIG QUIET)` for reasons I could not yet figure
out.

Closes #14555
2024-08-20 11:38:40 +02:00
Viktor Szakats
cd683f9071
cmake: add find_package() missing from USE_MSH3 option
The original patch added the Find module and CMake option. But the logic
missed a `find_package(MSH3)` call to use that Find module, leaving the
referenced `MSH3_INCLUDE_DIRS`, `MSH3_LIBRARIES` variables undefined.

Blind fix.

Follow-up to 37492ebbfa #8517

Closes #14609
2024-08-20 11:38:27 +02:00
Sam Jessup
d8cefac245
cf-socket: prevent KEEPALIVE_FACTOR being set to 1000 for Windows
Fixes #14368
Reported-by: feelingseas on github
Closes #14606
2024-08-20 09:24:12 +02:00
Daniel Stenberg
26e9d3a896
curl: find curlrc in XDG_CONFIG_HOME without leading dot
If XDG_CONFIG_HOME is set, look for XDG_CONFIG_HOME/curlrc - without a
leading dot in the filename.

Fixes #12129
Reported-by: Jat Satiro
Closes #14600
2024-08-20 08:44:44 +02:00
Viktor Szakats
96b9027f16
GHA/windows: unblock TFTP MQTT WebSockets SMTP FTP tests
Run them now. Also ignore results for now.

Closes #14607
2024-08-20 00:50:24 +02:00
Viktor Szakats
c555ab469d
cmake: limit pkg-config to UNIX and MSVC+vcpkg by default
Limits `pkg-config` to UNIX and MSVC with vcpkg, by default. Compared to
curl 8.9.1, this unlocks `pkg-config` on MSVC with vcpkg.

This condition might be updated in the future depending on where
`pkg-config` can be useful without breaking things. (e.g. to non-cross
MINGW, or all MINGW).

In the meantime everyone is free to override the default and test their
build with `pkg-config` by setting the `CURL_USE_PKGCONFIG=ON` CMake
option.

Closes #14575
2024-08-20 00:50:11 +02:00
Viktor Szakats
211cbcb4f6
cmake: rename Find modules
- `FindCARES`   -> `FindCares`
- `FindLibPSL`  -> `FindLibpsl`
- `FindLibSSH2` -> `FindLibssh2`
- `FindQUICHE`  -> `FindQuiche`
- `Findrustls`  -> `FindRustls`

Our convention for naming Find modules (the part after the `Find`
prefix, also called as 'package name') is:

Always start with uppercase. Follow with lowercase, unless there is
a clear preference for a stylized name. E.g. the project itself uses it
that way with a matching `<Name>Config.cmake` file, or we use it that
way elsewhere, or the name is an acronym.

Ref: #14580

Closes #14601
2024-08-20 00:50:10 +02:00
Viktor Szakats
3a2e47afb7
cmake: fix Find module and package names
- fix BearSSL warning about name mismatch.
- fix Nettle Find module not found on Linux.
- tidy-up: drop quotes from a package name.

Package names must match case-sensitively to work on all platforms:
- `find_package(<NAME> ...)` in `CMakeLists.txt`.
- `CMake/Find<NAME>.cmake` filenames.
- `find_package_handle_standard_args(<NAME> ...` in Find modules.
- `message(STATUS "Found <NAME> ...` in Find modules.
  (to match the message shown by `find_package_handle_standard_args()`)

Closes #14599
2024-08-20 00:44:43 +02:00
Viktor Szakats
c5cb8e7c7e
tidy-up: spelling quiche and Rustls
Closes #14605
2024-08-20 00:44:10 +02:00
Viktor Szakats
0fb4e59269
tidy-up: adjust casing of project names (continued)
Replace remaining `LibSSH2` with `libssh2`.

Follow-up to 6343034dd1 #14160

Closes #14602
2024-08-20 00:44:04 +02:00
Daniel Stenberg
a5598b6fcb
pingpong: drain the input buffer when reading responses
As the data might be held by TLS buffers, leaving some and expecting to
get called again is error prone.

Reported-by: ralfjunker on github
Fixes #14201
Closes #14597
2024-08-19 23:31:38 +02:00
Daniel Stenberg
ca88235102
KNOWN_BUGS: Heimdal memory leaks
Closes #14446
Closes #14604
2024-08-19 23:21:51 +02:00
Daniel Stenberg
145f87b9e8
build: use -Wno-format-overflow
-Wformat-overflow is not a warning that we want enabled as it does not
help us. It can only bring us false positives since it warns on bad uses
of sprintf and vsprintf ("that might overflow the destination buffer").
Two functions we explicitly ban in curl code.

The only way this flag triggers warnings in curl code is false positives
for functions we have marked with the CURL_PRINTF() macro.

Further: it seems -Wformat-trunaction option might in turn also enable
-Wformat-overflow, so if this second option is used, we need to
explicitly set -Wno-format-overflow - not just skip setting
-Wformat-overflow.

Reported-by: Viktor Szakats
Fixes #14168
Closes #14598
2024-08-19 23:20:45 +02:00
Viktor Szakats
c2e814f8d7
cmake/FindNettle: log message when found via pkg-config
The message mimics the CMake-native message (by
`find_package_handle_standard_args()`), with the header path and version number.

Closes #14596
2024-08-19 14:09:14 +02:00
Viktor Szakats
9fc2d7b8de
cmake: adjust GSSAPI option description
krb5 also builds with CMake, not only Heimdal.

Ref: 558814e16d

Closes #14595
2024-08-19 14:09:14 +02:00
Viktor Szakats
12399737c2
CI/azure: disable parallel tests, allow IDN tests
They started show the similar flakiness as the GHA ones after enabling
parallel tests (`-j2`) by default.

Example flaky run:
https://dev.azure.com/daniel0244/curl/_build/results?buildId=24763&view=results

Ubuntu:
```
FAIL 137: 'FTP download without size in RETR string' FTP, RETR, --data-binary
FAIL 336: 'FTP range download when SIZE doesn't work' FTP, PASV, TYPE A, RETR
FAIL 975: 'HTTP with auth redirected to FTP allowing auth to continue' HTTP, FTP, --location-trusted
FAIL 1378: 'FTP DL, file without Content-Disposition inside, using -o fname' FTP, RETR
```

MSYS2 mingw32:
```
FAIL 1501: 'FTP with multi interface and slow LIST response' FTP, RETR, multi, LIST, DELAY
```

MSYS2 mingw64:
```
FAIL 1501: 'FTP with multi interface and slow LIST response' FTP, RETR, multi, LIST, DELAY
```

Follow-up to 0324d557e4 #11510

Closes #14593
2024-08-19 14:09:14 +02:00
Viktor Szakats
47849be5d5
cmake/FindNettle: skip pkg-config for custom configs
If either `NETTLE_INCLUDE_DIR` or `NETTLE_LIBRARY` is set to customize
the `nettle` dependency, skip `pkg-config` and use the CMake-native
detection to honor these custom settings.

Closes #14584
2024-08-19 14:09:14 +02:00
Jan Venekamp
5b2a659ea6
mbedtls: fix setting tls version
TLS max values lower than 1.2 were automatically set to 1.2. Other SSL
backends (that dropped TLS 1.0 and 1.1) do not do that.

Closes #14588
2024-08-19 10:59:01 +02:00
Jan Venekamp
ff94698d31
wolfssl: fix setting tls version
The value CURL_SSLVERSION_TLSv1_0 was unsupported.

Closes #14587
2024-08-19 10:50:31 +02:00
Jan Venekamp
38fa458e51
rustls: fix setting tls version
The value CURL_SSLVERSION_TLSv1_0 was unsupported.

Closes #14586
2024-08-19 10:48:05 +02:00
Jan Venekamp
7a7c7a8995
bearssl: fix setting tls version
Previously version_max was ignored.

Closes #14585
2024-08-19 10:46:58 +02:00
Daniel Stenberg
73f62acaa2
RELEASE-NOTES: synced 2024-08-18 22:57:37 +02:00
Viktor Szakats
dcf5a5383c
cmake: fix cmakelint warnings
- keep line lengths below 132 characters.
- fix two "weird indentation" warnings.

Reported-by: Dan Fandrich
Bug: #14580

Closes #14583
2024-08-18 22:53:09 +02:00
Viktor Szakats
3e60f174ee
cmake: tidy up more in Find modules
- add `NAMES` where missing.
- document input variables (including deprecated ones.)
- comment cleanups.
- FindWolfSSL: drop stray `QUIET` from `pkg_check_modules()`.
  (`QUIET` may be re-added for all modules in the future.)

Closes #14579
2024-08-18 22:53:09 +02:00
Viktor Szakats
c57d3aeb55
appveyor: drop uploading artifacts
Uploading artifacts sometimes results in this error:
```
Uploading artifacts...
[1/1] _bld\src\curl.exe (2,022,912 bytes)...100%
Error uploading artifact to the storage: Remote server returned 503: Service Temporarily Unavailable
```
Ref: https://ci.appveyor.com/project/curlorg/curl/builds/50424126/job/e4envval6xkicv1i#L123

The artifacts are also probably not useful to upload for every run. Also
note that they were missing external DLL dependencies.

Leave the logic there commented, to make it easy to enable as needed for
debugging or testing artifacts locally.

Closes #14581
2024-08-18 12:22:53 +02:00
Viktor Szakats
1d29246534
cmake: tidy up around ngtcp2 and wolfSSL
- fix to add the `m` library without path.
  Follow-up to 8577f4ca08 #14343
  Authored-by: Tal Regev
  Fixes #14549

- move `m` library detection to wolfSSL Find module.
  `m` is necessary for wolfSSL (wolfcrypt) library functions called by
  `libngtcp2_crypto_wolfssl`.
  Follow-up to 8577f4ca08 #14343

- fix comment header about supported `COMPONENT` names.

- quote strings.

- lowercase local variables.

Closes #14576
2024-08-18 11:17:20 +02:00
Viktor Szakats
24889acbfb
cmake: do not unset the deprecated mixed-case variables
To avoid interference with the calling env.

(Keep unsetting for the DIRS/DIR cases in BearSSL and mbedTLS, because
the deprecated variables play a new role in the detection.)

Follow-up to 9fbda4ca75 #14574
2024-08-17 22:50:16 +02:00
Viktor Szakats
9fbda4ca75
cmake: rename wolfSSL and zstd config variables to uppercase
To match with other config variables and other projects.

Rename these CMake configuration variables:
- `WolfSSL_INCLUDE_DIR` -> `WOLFSSL_INCLUDE_DIR`
- `WolfSSL_LIBRARY`     -> `WOLFSSL_LIBRARY`
- `Zstd_INCLUDE_DIR`    -> `ZSTD_INCLUDE_DIR`
- `Zstd_LIBRARY`        -> `ZSTD_LIBRARY`

The old values continue to work, with a warning suggesting the new name.

Also:
- add similar warnings for earlier renames for mbedTLS and BearSSL.
- rename internal variables `PC_Zstd_*` to uppercase.

Follow-up to db39c668a8 #14542

Closes #14574
2024-08-17 21:23:51 +02:00
Daniel Stenberg
47a4864718
location: fix typo
Follow-up to 5fcf96930e
Bug: https://github.com/curl/curl/pull/14471#pullrequestreview-2244131475
Reported-by: Joshix-1 on github
2024-08-17 21:04:34 +02:00
XYenon
5fcf96930e
docs: add description of effect of --location-trusted on cookie
Closes #14471
2024-08-17 19:44:39 +02:00
Jan Venekamp
88727f7ed0
docs: improve cipher options documentation
Closes #14407
2024-08-17 11:14:21 +02:00
renovate[bot]
b2488afb18
GHA: update github/codeql-action digest to 429e197
Closes #14425
2024-08-17 11:03:09 +02:00
Viktor Petersson
6fc66e1676
SECURITY: mention OpenSSF best practices gold badge
Closes #14319
2024-08-17 11:01:16 +02:00
Justin Maggard
88cae14550
mbedtls: add more informative logging
After TLS handshare, indicate which TLS version was negotiated in
addition to the cipher in the handshake completed log message.

Also use the verify callback for certificate logging and collection.
This allows things to work even when MBEDTLS_SSL_KEEP_PEER_CERTIFICATE
is disabled in the mbedtls library.

And lastly, catch certificate validation errors later so we can give the
user more informative error messages that indicate what the failure was
from certificate validation.

Tested on both current LTS versions (2.28 and 3.6).

Closes #14444
2024-08-17 10:57:45 +02:00
renovate[bot]
2c4d04c4d3
GHA: update dependency gnutls/gnutls to v3.8.7
Closes #14554
2024-08-17 10:53:41 +02:00
Stefan Eissing
a58b50fca6
transfer: Curl_sendrecv() and event related improvements
- Renames Curl_readwrite() to Curl_sendrecv() to reflect that it
  is mainly about talking to the server, not reads or writes to the
  client. Add a `nowp` parameter since the single caller already
  has this.
- Curl_sendrecv() now runs all possible operations whenever it is
  called and either it had been polling sockets or the 'select_bits'
  are set.
  POLL_IN/POLL_OUT are not always directly related to send/recv
  operations. Filters like HTTP/2, QUIC or TLS may monitor reverse
  directions. If a transfer does not want to send (KEEP_SEND), it
  will not do so, as before. Same for receives.
- Curl_update_timer() now checks the absolute timestamp of an expiry
  and the last/new timeout to determine if the application needs
  to stop/start/restart its timer. This fixes edge cases where
  updates did not happen as they should have.
- improved --test-event curl_easy_perform() simulation to handle
  situations where no sockets are registered but a timeout is
  in place.
- fixed bug in events_socket() that complained about removing
  a socket that was unknown, when indeed it had removed the socket
  just before, only it was the last in the list
- fixed conncache's internal handle to carry the multi instance
  (where the cache has one) so that operations on the closure handle
  trigger event callbacks correctly.
- fixed conncache to not POLL_REMOVE a socket twice when a conneciton
  was closed.

Closes #14561
2024-08-17 10:52:53 +02:00
Viktor Szakats
432f2fd9ac
cmake: sync up version detection in Find modules
- use the same pattern across all Find modules:
  - verify if the version header exists before reading it.
  - use a single regex per lookup.
  - sync regexes between Find modules.
  - use generic temporary variable names.
  - improve readability.
  - make it simpler to transition to new CMake syntax in the future:
    ```cmake
    file(STRINGS "${CARES_INCLUDE_DIR}/ares_version.h" _version_str REGEX "<...>")
    unset(_version_str)
    set(CARES_VERSION "${CMAKE_MATCH_1}")
    ```
    Ref: https://cmake.org/cmake/help/latest/policy/CMP0159.html#policy:CMP0159

- fix zstd version detection to be CMake 3.7 compatible.
  Required 3.9 before this patch, for the `CMAKE_MATCH_<n>` feature.
  Follow-up to c5d506e9bb #12200

Follow-up to 4e2f3641f8 #14548

Closes #14572
2024-08-17 10:33:26 +02:00
Viktor Szakats
d8de4806e1
cmake: tidy-up continues
- move variable dump to a GHA foldable group.
- minimize scope for an include().
- rename `HIDES_CURL_PRIVATE_SYMBOLS` to `CURL_HIDES_PRIVATE_SYMBOLS`,
  to keep it in the curl namespace.
- drop quotes from a version number.
- add missing `Makefile.inc` var refs to comment.
- FindNGTCP2: rename internal var to underscore/lowercase.
- FindBearSSL, FindGSS: whitespace.

Closes #14571
2024-08-17 00:32:38 +02:00
Viktor Szakats
f3a03df6a1
cmake: revert to pkg_check_modules()
Prefer `pkg_check_modules()` over `pkg_search_module()`.

`pkg_check_modules()` logs a line when there is a hit, and also warnings
if a sub-dependency is missing. In `QUIET` mode, both are silent.

The extra info is useful to see if a detection happened via
`pkg-config`.

Keep `pkg_search_module()` in `FindGSS`. We pass two dependencies
there and we want to keep stopping on the first one.

Partially reverts c2889a7b41 #14388

Closes #14573
2024-08-17 00:31:52 +02:00
Viktor Szakats
4beb236478
cmake: fixup variable reference in FindZstd
Follow-up to 4e2f3641f8 #14548
2024-08-16 19:10:46 +02:00