34875 Commits

Author SHA1 Message Date
Viktor Szakats
627e9816ff
cmake: add custom command scripts as dependencies where missing
Also some formatting.

Closes #16835
2025-03-26 15:31:53 +01:00
Daniel Stenberg
39931f7fbe
RELEASE-NOTES: synced 2025-03-26 07:54:09 +01:00
Viktor Szakats
4af9d75783
GHA/non-native: revert to bare builds for Android 21
vcpkg requires Android 28 by default after a recent update that's being
deployed onto CI runs (with `libiconv:arm64-android@1.18#1`).

Revert to bare, no-ssl, no-psl configuration for Android 21 jobs to make
them work again.

Bug: https://github.com/curl/curl/pull/16824#issuecomment-2750912507
Ref: https://github.com/microsoft/vcpkg/pull/44424#issuecomment-2753027630

Closes #16832
2025-03-26 03:31:10 +01:00
Viktor Szakats
1447726663
cmake: document SH_EXECUTABLE option
Closes #16830
2025-03-26 00:54:04 +01:00
Daniel Stenberg
7d1a5c6bdf
tests: remove base64 encoded sections
The base64 sections were added in a time when there was no other way to
do these tests. Now we remove them again to make less "hidden" data in
the test suite.

For several test cases we instead then provide a sequence of binary
octets which is not really better, but I have tried to minimize them.

Closes #16816
2025-03-25 16:47:48 +01:00
Daniel Stenberg
9a7baa34cc
mqtt: cleanups
- make a variable const
- use calloc instead of malloc + memset
- fix comment language
- improve error messages

Closes #16826
2025-03-25 09:18:08 +01:00
Daniel Stenberg
b2078c0e1c
mqtt: convert sendleftovers to dynbuf
Avoid frequent strdups/free calls, including the double-free risk.

Reported-by: Ronald Crane
Closes #16823
2025-03-25 08:22:05 +01:00
Ben Bodenmiller
38b94c80f3
SSLCERTS: list support for SSL_CERT_FILE and SSL_CERT_DIR
Closes #16762
2025-03-25 08:20:39 +01:00
tiymat
d055a01ce9
scripts: update completion.pl to parse options from docs
Reported-by: kpcyrd on github
Fixes #16072
Closes #16789
2025-03-24 23:48:26 +01:00
Gabriel Marin
c329321bf9
easy: add 'populate_fds' func to reduce size of 'wait_or_timeout'
Closes #16820
2025-03-24 23:46:52 +01:00
Daniel McCarney
088f0e6a5b
rustls: tidy up
Closes #16796
2025-03-24 23:45:02 +01:00
Jay Satiro
e601668154 openssl: fix crash on missing cert password
- Return 0 for password length if OpenSSL is expecting a certificate
  password but the user did not provide one.

Prior to this change libcurl would crash if OpenSSL called the certificate
password callback in libcurl but no password was provided (NULL).

Reported-by: Roman Zharkov

Fixes https://github.com/curl/curl/issues/16806
Closes https://github.com/curl/curl/pull/16807
2025-03-24 18:22:53 -04:00
Daniel Stenberg
18f04faef9
asyn-thread: minor cleanups
- Use 'td->' in more places instead of the full thing.

- Remove an assert that is always true since the extra dynamic malloc
  was removed

- Ignore Curl_ares_perform() errors to prioritize the "real" resolver
  info and leave RR as "less important"

Closes #16808
2025-03-24 22:54:39 +01:00
Dave Nicolson
ed0966643d
tool_operate: fix pluralization of seconds
Closes #16751
2025-03-24 22:44:09 +01:00
Stefan Eissing
a95b291ec0
conncache: eliminate cpool's diconnect callback
The callback, provided from url.c did the work that the cshutdn
functionality also implemented. Remove it.

Change some DEBUGF(infof()) to CURL_TRC_M().

Closes #16810
2025-03-24 22:41:53 +01:00
renovate[bot]
ac3c353747
GHA: update mbedtls to v3.6.3
Closes #16822
2025-03-24 22:36:05 +01:00
Viktor Szakats
579625efa5
test615: fix for Cygwin, unignore in CI
Setting a server-side file read-only by `chmod 0444` has does not
prevent overwriting it via SFTP upload (as tested in CI).

Fix it by setting its MS-DOS read-only attribute in addition. It
requires the Cygwin tool `chattr`.

Also unignore in CI.

Fixes:
```
test 0615...[SFTP put remote failure]
curl returned 0, when expecting 9
 615: exit FAILED
=== Start of file stderr615
   % Total    % Received % Xferd  Average Speed   Time    Time     Time  Current
                                  Dload  Upload   Total   Spent    Left  Speed
   0     0    0     0    0     0      0      0 --:--:-- --:--:-- --:--:--     0
 100    30    0     0  100    30      0     93 --:--:-- --:--:-- --:--:--    95

 100    30    0     0  100    30      0     92 --:--:-- --:--:-- --:--:--    92
=== End of file stderr615
```
Ref: https://github.com/curl/curl/actions/runs/14037991918/job/39300723214#step:12:1269

Closes #16818
2025-03-24 16:49:57 +01:00
Viktor Szakats
89101fabf8
GHA/windows: dl-mingw PATH follow-up
Follow-up to 468bfc26183cc4e65b2494acf82d256571cb8178 #16813

Closes #16817
2025-03-24 15:34:00 +01:00
Daniel Stenberg
76d7483fe5
INFRASTRUCTURE.md: add IRC and Matrix details
Closes #16809
2025-03-24 15:21:59 +01:00
Viktor Szakats
ac6f7145a9
GHA/windows: use a pure Cygwin environment
Use the `PATH` `/usr/bin` to avoid any Windows system or 3rd-party tool
installed on the runner machine that may interfere with or add undesired
dependencies to the builds and tests.

Follow-up to d838d434305bd97c0ce5488f24945beff878adc9 #16465
Ref: #16437

Closes #16814
2025-03-24 12:41:52 +01:00
Viktor Szakats
468bfc2618
GHA/windows: boost dl-mingw and cygwin install performance
Install on drive `D:` which has much better write performance than `C:`,
on GitHub Windows runner machines.

- It's bringing down `dl-mingw` installation steps to 5-15s per job,
  from 15s-130s before this patch.

- Saving 30-90s per job in the Cygwin install step.

The before values were fluctuating, but it seems reasonable to expect
saving at least a couple of minutes for each workflow run.

Closes #16813
2025-03-24 12:37:44 +01:00
Viktor Szakats
5cce329873
libssh2: minor tidy-ups
Fix indentation and drop an unnecessary assigment.

Closes #16812
2025-03-24 12:37:44 +01:00
Viktor Szakats
be21c95740
GHA/windows: install OpenSSH-Windows manually for transparency
To have the current latest version, and to avoid the stale, misleading
versions installed by Chocolatey. It also installs transparently, faster,
and making the source of the binaries clear. Install on drive `D:` for
best performance.

After much detective work it turns out that the OpenSSH Windows versions
installed by Chocolatey aren't what they seem:

- The latest pre-release named 9.5.0-beta20240403:
  https://community.chocolatey.org/packages/openssh/9.5.0-beta20240403
  is in reality 8.6.0.0p1-Beta from 2021-05-27:
  https://github.com/PowerShell/Win32-OpenSSH/releases/download/V8.6.0.0p1-Beta/OpenSSH-Win64.zip

- The latest "stable" version 8.0.0.1 is in reality:
  https://community.chocolatey.org/packages/openssh/8.0.0.1
  is in reality 8.0.0.0p1-Beta:
  https://github.com/PowerShell/Win32-OpenSSH/releases/download/v8.0.0.0p1-Beta/OpenSSH-Win64.zip

Ref: https://github.com/curl/curl/pull/16803#issuecomment-2746365654
Follow-up to 67a7775d1233d702964bff9a0a6b5b9fa036c47a #16704
Follow-up to 0ec72c1ef8d87a29bf2eaa5e36ab173147a4d015 #16672

Closes #16811
2025-03-24 12:37:44 +01:00
Viktor Szakats
e7944fb3da
GHA: extend clang-tidy jobs with more build options, add Windows job
- linux: wolfssl, wolfssh (replacing libssh2), ech, kerberos/GSSAPI,
  ssls-export (libssh2 remains tested on macos.)

- macos: brotli, zstd, c-ares, gnutls, mbedtls, gsasl, rtmp, ssls-export

- windows: new job with schannel, sspi, winidn, winldap, ssls-export

- unit3205: fix/silence remaining NULL dereferences.

Commits fixing the issues found:
cbbccb8b3aa357926131375e7082b2579fa3fe82 #16766
554e4c14bee82a604e08d51767f49fe11580e700 #16777

Closes #16764
2025-03-24 12:37:43 +01:00
Stefan Eissing
c712effda3
multi_ev: fixes regarding connection shutdowns
When handling connections (not transfers) related events, always use the
passed `conn` and not `data->conn` as the transfer does not need to have
the same connection.

Fix handling of conn pollset diffs to disregard the transfer.

Closes #16782
2025-03-24 10:55:40 +01:00
Viktor Szakats
7991b5a85e
memdebug: drop dynamic allocation from curl_dbg_log()
Closes #16745
2025-03-24 10:22:58 +01:00
Viktor Szakats
554e4c14be
windows: fix issues detected by clang-tidy, and some more
- digest_sspi: memory leak.
- digest_sspi: free buffers on `calloc()` fail.
  (not detected by clang-tidy)
- schannel_verify: avoid a `NULL` `alt_name_info`.
- schannel: fix potential `NULL` deref for `backend->cred`.
- schannel: fix uninitialized result value.
  Follow-up to 7f4c358541fdadcf29ba20bcdff71c5554e5f69c #3197
- schannel: drop unused assigment.
- tool_doswin: drop unused assigment.
- testutil: fix memory leak on error.
- testutil: fix memory leak on non-error.
  (not detected by clang-tidy)

Cherry-picked from #16764
Closes #16777
2025-03-24 10:15:08 +01:00
Viktor Szakats
c48c4914a9
tests: build non-debug unit tests with autotools, run them
Before this patch, autotools disabled building unit tests for
non-debug-enabled (`DEBUGBUILD`) builds. runtests skipped running this
combination, though they were built in cmake builds. There seems to be
no technical reason anymore for these restrictions. This patch removes
them, allowing to build and run unit tests for non-debug-enabled builds.

To improve unit test build and run coverage.

- autotools: do not disable building unit tests for non-debug-enabled
  build. Bringing behavior closer to cmake builds. (There are still
  exceptions in autotools, something for another PR)

- runtests: run unit tests for non-debug-enabled builds.
  It extends coverage by 50 tests.

- `lib/altsvc.c`: fix to include `CURL_TIME` support in libcurlu, for
  unit tests. It fixes test 1654, and syncs `CURL_TIME` behavior with
  test 1660 and `lib/hsts.c`.

Ref: 10a7d05be3921b9c9762477a18d41aff232d94d8
Ref: fc8e0dee3045658f293452121f5290d81ba3aa1e #13694
Ref: 99f78cbf6ed376391f519de18d289a6924228591 #16770

Closes #16771
2025-03-24 10:15:05 +01:00
Viktor Szakats
cbbccb8b3a
mbedtls: re-enable an error check
While adding support for key blobs, a check and error code update moved
after some logic, resulting in the updated code not checked anymore.

Detected by clang-tidy:
```
lib/vtls/mbedtls.c:768:7: error: Value stored to 'ret' is never read [clang-analyzer-deadcode.DeadStores,-warnings-as-errors]
  768 |       ret = MBEDTLS_ERR_PK_TYPE_MISMATCH;
      |       ^
```
Ref: https://github.com/curl/curl/actions/runs/13953249156/job/39057979349?pr=16764#step:12:178

Regression from 05e0453050def7323fbc8210886f909819edcc5f #7157
Cherry-picked from #16764
Closes #16766
2025-03-24 10:15:02 +01:00
Viktor Szakats
9465327084
cmake: fix clang-tidy builds to verify tests, fix fallouts
- cmake: disable test bundles for clang-tidy builds.
  clang-tidy ignores #included .c sources, and incompatible with unity
  and bundles. It caused clang-tidy ignoring all test sources. It also
  means this is the first time tests sources are checked with
  clang-tidy. (autotools doesn't run it on tests.)

- cmake: update description for `CURL_TEST_BUNDLES` option.

- fix tests using special `CURLE_*` enums that were missing from
  `curl/curl.h`. Add them as reserved codes.

- fix about ~50 other issues detected by clang-tidy: unchecked results,
  NULL derefs, memory leaks, casts to enums, unused assigments,
  uninitialized `errno` uses, unchecked `open`, indent, and more.

- drop unnecessary casts (lib1533, lib3207).

- suppress a few impossible cases with detailed `NOLINT`s.

- lib/escape.c: drop `NOLINT` no longer necessary.
  Follow-up to 72abf7c13a479edcde80afa60faad3f35f672c0b #13862 (possibly)

- extend two existing `NOLINT` comments with details.

Follow-up to fabfa8e4024473035b3e5c3c30c330be726d9bb4 #15825

Closes #16756
2025-03-24 10:14:58 +01:00
Viktor Szakats
efa65b24ae
test613: make it pass on Windows, fix postprocess, unignore in CI
- on native Windows (also when using MSYS2 openssh), the group and other
  permissions do not end up as requested by Perl's chmod:
  ```diff
  --- log/8/check-expected
  +++ log/8/check-generated
  @@ -1,3 +1,3 @@
   d?????????    N U         U               N ???  N NN:NN asubdir[LF]
  --rw?rw?rw?    1 U         U              37 Jan  1  2000 plainfile.txt[LF]
  +-rw?r-?r-?    1 U         U              37 Jan  1  2000 plainfile.txt[LF]
   -r-?r-?r-?    1 U         U              47 Dec 31  2000 rofile.txt[LF]
  ```
  Ref: https://github.com/curl/curl/actions/runs/14004029192/job/39215359241?pr=16781#step:15:1596
  Fix it by ignoring group and other attributes.

- fix failing postprocess cleanup by making the read-only test file
  writeable again before deleting it. Fixing:
  ```
  Directory not empty at ../../tests/libtest/test613.pl line 83.
  ```
  (seen on Windows with Git for Windows `perl.exe`)

- unignore in GHA/windows.

Closes #16791
2025-03-24 10:05:17 +01:00
Viktor Szakats
d838d43430
runtests: use setfacl on Cygwin/MSYS, if present
To use a native Cygwin tool instead of the Windows `icacls`. It allows
running under Cygwin/MSYS without Windows system folders in the `PATH`.

Also: fix indentation and tidy up syntax of the `icacls` branch.

Note: As of this commit, these `setfacl` and `icacls` calls are not
necessary for a successful CI run. This includes OpenSSH for Windows
tests, that aren't run by default. Keep them anyway, because locally
they may be necessary depending on environment.

Reported-by: Brian Inglis
Fixes #16437
Ref: #16803
Closes #16465
2025-03-24 10:05:17 +01:00
Weng Xuetian
46c6ca789e
asyn-thread: check thread_data->init in Curl_resolver_getsock
resolver may call destroy_async_data after the name is resolved and
corresponding socketpair is already closed at this point. Any following
call to Curl_resolver_getsock should not set the fd.

Fixes #16799
Closes #16802
2025-03-24 10:03:12 +01:00
Stefan Eissing
f68eae250b
conn: eliminate conn->now
it was only used in pingpong.c to check if the overall transfer has
timed out and we do that with `Curl_timeleft()` in all other places.

Closes #16793
2025-03-24 09:57:38 +01:00
Stefan Eissing
ec4e2cd15d
conncache: eliminate conn->destination_len as premature optimization
Closes #16792
2025-03-24 09:56:13 +01:00
Stefan Eissing
646b2d6ca2
dynbuf: assert init on free
Add a DEBUGASSERT() in Curl_dyn_free() that checks that Curl_dyn_init()
has been performed before.

Fix code places that did it wrong.

Fixes #16725
Closes #16775
2025-03-24 09:53:40 +01:00
Viktor Szakats
cd7eb9e0f2
libssh2: fix to ignore known_hosts if SHA256 host public key is set
Syncing behavior with MD5 host public keys.

libcurl implemented to force a host key type for hosts is present in
`known_hosts`, and disabled this logic when an MD5 host public key is
explicitly set. libcurl later received support for SHA256 host public
keys. This update missed to extend the `known_hosts` logic with the new
key type.

This caused test 3022 to fail if a pre-existing `known_hosts` listed
the test server IP (127.0.0.1) with a non-RSA host key algo.

Follow-up to d1e7d9197b7fe417fb4d62aad5ea8f15a06d906c #7646
Follow-up to 272282a05416e42d2cc4a847a31fd457bc6cc827 #4747

Closes #16805
2025-03-24 09:47:59 +01:00
Viktor Szakats
581a7b07a6
test612: SCP rm the uploaded remote file (not the local source), unignore in CI
It accidentally worked on all CI-tested operating systems, except on
native Windows.

Fixing:
```
=== Start of file stderr612
   % Total    % Received % Xferd  Average Speed   Time    Time     Time  Current
                                  Dload  Upload   Total   Spent    Left  Speed
[...]
 curl: (21) rm command failed: Operation failed
```
Ref: https://github.com/curl/curl/actions/runs/14004029192/job/39215359241?pr=16781#step:15:1424

Also remove this test from the ignore list in GHA/windows.

Closes #16801
2025-03-24 09:47:59 +01:00
Daniel Stenberg
8ab3deedf4
RELEASE-NOTES: synced 2025-03-24 09:33:30 +01:00
Viktor Szakats
7d64601a8c
sshserver.pl: adjust AuthorizedKeysFile2 cutoff version
To avoid in `ssh_server.log`:
```
reprocess config line 6: Deprecated option AuthorizedKeysFile2
```
with openssh 7.1

Closes #16787
2025-03-23 20:26:41 +01:00
Viktor Szakats
72b163c301
runtests: support running tests under wine or qemu
To run curl, tests and servers via `wine`:
```shell
export CURL_TEST_EXE_RUNNER=wine
```
runtests prefixes commands with the specified runner. For systems where
this isn't automatic or supported, e.g. macOS.

Closes #16785
2025-03-23 20:26:38 +01:00
Viktor Szakats
b214acbb0c
sshserver: fix excluding obsolete client config lines
It was already excluded for OpenSSH-Windows. Extend it to all OpenSSH
above v7.3. Syncing up this logic with the sshd server config.

Fixing, in `sftp_server.log`:
```
log/server/curl_sftp_config line 33: Unsupported option "rhostsrsaauthentication"
log/server/curl_sftp_config line 34: Unsupported option "rsaauthentication"
```

`no` has been the default for these since OpenSSH 3.3 (2002-06-21).

Closes #16784
2025-03-23 20:26:35 +01:00
Yedaya Katsman
fd6e6e8c7a
secureserver.pl, genserv.sh: cleanups
stunnel: print configuration actually in file
Instead of duplicating all the logic of writing the stunnel
configuration, and having to keep it up to date, when running in verbose
mode print the actual contents of the file after writing it.

certs/genserv.sh: Fix typo in error message

Closes #16783
2025-03-23 20:26:30 +01:00
Viktor Szakats
89f306ae40
runtests: fix test key format for libssh2 WinCNG (and others)
SFTP/SCP tests were failing in CI with WinCNG libssh2 since we first
added such job. With `curl: (67) Authentication failure`.

The reason is that the default `ssh-keygen` RSA private key format
changed to OpenSSH (RFC4716) in 2018. libssh2 does not support this
format with some of its crypto backends.

Fix it by generating keys explicitly in PEM format as necessary via
the `-m` option. This format is universally recognized for RSA keys.

2018-08-24: https://www.openssh.com/txt/release-7.8: OpenSSH format becomes default
2010-08-23: https://www.openssh.com/txt/release-5.6: `-m` option first supported

This fixed the auth issue, just to reveal a known flakiness issue in
libssh2 + WinCNG, causing:
```
curl: (2) Failure establishing ssh session: -8, Unable to exchange encryption keys
```
Ref: https://github.com/curl/curl/actions/runs/14000494428/job/39205633258?pr=16781#step:15:1796
Tracked here: https://github.com/libssh2/libssh2/issues/804
Mitigated in libssh2 tests by retrying them.

Due to this, keep ignoring these test results.

Also:
- add an env to customize key format: `CURL_TEST_SSH_KEY_FORMAT`
- display the generated format in the log.
- GHA/linux: document the wolfSSH error code causing it to fail tests:
  ```
  curl: (79) wolfssh SFTP connect error -1051 / WS_MATCH_KEY_ALGO_E / cannot match key algo with peer
  ```

Follow-up to 4911e7af119c1b7efd46a742d47bca44832c3041 #16735
Follow-up to 0ec72c1ef8d87a29bf2eaa5e36ab173147a4d015 #16672
Follow-up to e53523fef07894991c69d907a7c7794c7ada4ff4 #14859
Follow-up to e26cbe20cbedbea0ca743dd33880517309315cb2 #13979

Closes #16781
2025-03-23 20:26:26 +01:00
Peter Kokot
dd0c9feb54
cmake: replace CMAKE_COMPILER_IS_GNUCC with CMAKE_C_COMPILER_ID
Variable `CMAKE_COMPILER_IS_GNUCC` is deprecated and
`CMAKE_C_COMPILER_ID` should be used instead.

Closes #16797
2025-03-23 00:06:25 +01:00
Peter Kokot
d03429ce03
cmake: replace exec_program() with execute_process()
The `exec_program()` is deprecated as of CMake 3.0.

This also removes the `rm_out` variable as it isn't used in the output.
In `execute_process()` the `ERROR_QUIET` and `OUTPUT_QUIET` resemble
the behavior of `exec_program(OUTPUT_VARIABLE)` behavior in this case.

Closes #16779
2025-03-23 00:06:25 +01:00
Samuel Henrique
23f6a1abfd tests: Remove unused variables
* tests/directories.pm:
  - %diroutput
* tests/ftpserver.pl:
  - %exit_signal
* tests/test1135.pl:
  - @syms
  - %doc
  - %rem
* tests/test1167.pl:
  - %doc
  - %rem
* tests/tftpserver.pl:
  - %fork

Closes #16798
2025-03-22 11:12:29 -07:00
Daniel McCarney
58881058ad
cmake: fix typo in ECH config error msg
Just a small typo I noticed while working on wiring up rustls-ffi vTLS
ECH support.

Closes #16786
2025-03-21 15:55:23 +01:00
Stefan Eissing
18b0a584cb
autotools: add dependency of libtest and unit bundles to Makefile.inc
So that changes in libtests and unit tests are picked up.

Closes #16763
2025-03-20 11:28:56 +01:00
Viktor Szakats
08553aba97
configure: fix ECH detection with MultiSSL
Detect OpenSSL and wolfSSL support independently. Pass detection if
either of them has support. Before this patch wolfSSL results overwrote
OpenSSL detection results when both backends were enabled.

Also fix output message when both of them support ECH.

Closes #16774
2025-03-20 03:31:24 +01:00