Commit Graph

32864 Commits

Author SHA1 Message Date
Viktor Szakats
b05dc7eb35
sectransp: fix HAVE_BUILTIN_AVAILABLE checks to not emit warnings
`HAVE_BUILTIN_AVAILABLE` is a curl macro set via autotools and cmake.
Like other `HAVE_`s it signals availability if defined.

SecureTransport code was specifically looking for the value 1, which
triggered compiler warnings when the feature was not present.

Replace the existing workaround of locally suppressing the compiler
warning with using `defined()`.

autotools:
```
767 | #if (CURL_BUILD_MAC_10_13 || CURL_BUILD_IOS_11) && HAVE_BUILTIN_AVAILABLE == 1
      |                                                    ^~~~~~~~~~~~~~~~~~~~~~
../../lib/vtls/sectransp.c: In function 'sectransp_connect_step1':
../../lib/vtls/sectransp.c:1140:52: error: "HAVE_BUILTIN_AVAILABLE" is not defined, evaluates to 0 [-Werror=undef]
 1140 | #if (CURL_BUILD_MAC_10_13 || CURL_BUILD_IOS_11) && HAVE_BUILTIN_AVAILABLE == 1
      |                                                    ^~~~~~~~~~~~~~~~~~~~~~
../../lib/vtls/sectransp.c:1240:52: error: "HAVE_BUILTIN_AVAILABLE" is not defined, evaluates to 0 [-Werror=undef]
 1240 | #if (CURL_BUILD_MAC_10_13 || CURL_BUILD_IOS_11) && HAVE_BUILTIN_AVAILABLE == 1
      |                                                    ^~~~~~~~~~~~~~~~~~~~~~
../../lib/vtls/sectransp.c: In function 'sectransp_connect_step2':
```
Ref: https://github.com/curl/curl/actions/runs/9815428701/job/27104448045#step:6:499

cmake gcc:
```
 1140 | #if (CURL_BUILD_MAC_10_13 || CURL_BUILD_IOS_11) && HAVE_BUILTIN_AVAILABLE == 1
      |                                                    ^~~~~~~~~~~~~~~~~~~~~~
/Users/runner/work/curl/curl/lib/vtls/sectransp.c:1240:52: error: "HAVE_BUILTIN_AVAILABLE" is not defined, evaluates to 0 [-Werror=undef]
 1240 | #if (CURL_BUILD_MAC_10_13 || CURL_BUILD_IOS_11) && HAVE_BUILTIN_AVAILABLE == 1
      |                                                    ^~~~~~~~~~~~~~~~~~~~~~
/Users/runner/work/curl/curl/lib/vtls/sectransp.c: In function 'sectransp_connect_step2':
/Users/runner/work/curl/curl/lib/vtls/sectransp.c:2231:51: error: "HAVE_BUILTIN_AVAILABLE" is not defined, evaluates to 0 [-Werror=undef]
 2231 | #if(CURL_BUILD_MAC_10_13 || CURL_BUILD_IOS_11) && HAVE_BUILTIN_AVAILABLE == 1
      |                                                   ^~~~~~~~~~~~~~~~~~~~~~
```
Ref: https://github.com/curl/curl/actions/runs/9815428701/job/27104445425#step:8:355

Cherry-picked from #14097
Closes #14122
2024-07-08 17:22:21 +02:00
Viktor Szakats
5fc61a37c1
examples: suppress deprecation warnings locally
Simplify making clean builds by silencing deprecation warnings inside
the example code where these may occur.

Drop related build tweaks/comments from GHA jobs.

Example warning:
```
curl/docs/examples/postit2-formadd.c:65:16: error: 'CURLFORM_COPYNAME' is deprecated: since 7.56.0. Use curl_mime_name() [-Werror=deprecated-declarations]
   65 |                CURLFORM_COPYNAME, "sendfile",
      |                ^~~~~~~~~~~~~~~~~
```
Ref: https://github.com/curl/curl/actions/runs/9841099503/job/27166970904#step:10:829

Closes #14123
2024-07-08 17:22:21 +02:00
Viktor Szakats
72341068a2
GHA/macos: bump parallel tests to -j5
Credit-to: Dan Fandrich
Cherry-picked from #11510 #14097
2024-07-08 15:39:46 +02:00
Viktor Szakats
338f5ae680
GHA/windows: usability improvements
- move `curl --version` into separate step.

- move configure log to separate step. Run on success, too.

- add step with `curl_config.h` dump (full and brief/sorted).

- make `autoreconf` a separate step.

- add each job configuration a short name.

- shorten job names.
  Dedupe/drop redundant info, introduce abbreviations:
  AM = autotools, CM = CMake, U = Unicode, R = Release, not -> `!`, etc.
  Instead of mentioning `debug`, mentioned when it's not.

- simplify `PATH` forming for MSVC jobs.
  It's sufficient to add the release binary directory of vcpkg, the debug one
  is redundant.
  Follow-up to e26cbe20cb #13979

- other minor tidy-ups.

Closes #14116
2024-07-08 14:19:54 +02:00
Viktor Szakats
2c15aa5765
GHA/macos: delete misplaced CFLAGS, drop redundant CMake option
With macOS there is a long-term struggle with deprecation warnings.
In curl they occur with LDAP, SecureTransport and in docs/examples.

There are three ways to fix them:
- by CFLAGS `-Wno-deprecated-declarations` as a workaround.
- by CFLAGS `-mmacosx-version-min` set to a version where the the
 feature was not deprecated.
- by CMake option `-DCMAKE_OSX_DEPLOYMENT_TARGET=`.

In GHA CMake jobs, all three were used, and `-mmacosx-version-min` was
set in a bogus way. Delete that bogus option, and delete the lone,
redundant CMake option too.

In a future commit I might replace the suppression option to properly
setting the target OS.

Follow-up to dfdd978f7c #13491
Cherry-picked from #14097
2024-07-08 14:16:22 +02:00
Viktor Szakats
db135f8d72
macos: add workaround for gcc, non-c-ares, IPv6, compile error
Apple macOS SDK 13.0 and later are increasingly incompatible with gcc,
which started causing CI errors with the 20240701.9 revision of the
`macos-latest` (= `macos-14-arm64`) runner image.

This error is happening inside an Apple SDK header. We use the header
for calling a function in a resolver-related hack, in non-c-ares, IPv6
builds. You can avoid the problem by using c-ares or disabling IPv6
(or using clang, llvm, or a compatible gcc + SDK combination).

This patch fixes affected builds by declaring the ncessary framework
function manually, and not including the problematic header.

This workaround is ugly, doesn't cover all combinations, and fragile.

Other options are to disable this resolver-related hack for GCC, or to
replace it with a solution that doesn't rely on Apple SDK.

If you are aware of a stable fix or workaround, let us know.

gcc 12.4.0 + macOS SDK 14.0 (Xcode 15.0.1) error example:
```
In file included from /Applications/Xcode.app/Contents/Developer/Platforms/MacOSX.platform/Developer/SDKs/MacOSX14.0.sdk/System/Library/Frameworks/CoreFoundation.framework/Headers/CoreFoundation.h:54,
                 from /Applications/Xcode.app/Contents/Developer/Platforms/MacOSX.platform/Developer/SDKs/MacOSX14.0.sdk/System/Library/Frameworks/SystemConfiguration.framework/Headers/SCDynamicStoreCopySpecific.h:30,
                 from /Users/runner/work/curl/curl/lib/macos.c:33,
                 from /Users/runner/work/curl/curl/build/lib/CMakeFiles/libcurl_shared.dir/Unity/unity_0_c.c:244:
/Applications/Xcode.app/Contents/Developer/Platforms/MacOSX.platform/Developer/SDKs/MacOSX14.0.sdk/System/Library/Frameworks/CoreFoundation.framework/Headers/CFUserNotification.h:126:1: error: attributes should be specified before the declarator in a function definition
  126 | CF_INLINE CFOptionFlags CFUserNotificationCheckBoxChecked(CFIndex i) API_AVAILABLE(macos(10.0)) API_UNAVAILABLE(ios, watchos, tvos) {return ((CFOptionFlags)(1UL << (8 + i)));}
      | ^~~~~~~~~
/Applications/Xcode.app/Contents/Developer/Platforms/MacOSX.platform/Developer/SDKs/MacOSX14.0.sdk/System/Library/Frameworks/CoreFoundation.framework/Headers/CFUserNotification.h:127:1: error: attributes should be specified before the declarator in a function definition
  127 | CF_INLINE CFOptionFlags CFUserNotificationSecureTextField(CFIndex i) API_AVAILABLE(macos(10.0)) API_UNAVAILABLE(ios, watchos, tvos) {return ((CFOptionFlags)(1UL << (16 + i)));}
      | ^~~~~~~~~
/Applications/Xcode.app/Contents/Developer/Platforms/MacOSX.platform/Developer/SDKs/MacOSX14.0.sdk/System/Library/Frameworks/CoreFoundation.framework/Headers/CFUserNotification.h:128:1: error: attributes should be specified before the declarator in a function definition
  128 | CF_INLINE CFOptionFlags CFUserNotificationPopUpSelection(CFIndex n) API_AVAILABLE(macos(10.0)) API_UNAVAILABLE(ios, watchos, tvos) {return ((CFOptionFlags)(n << 24));}
      | ^~~~~~~~~
```
Ref: https://github.com/curl/curl/actions/runs/9787982387/job/27025351601?pr=14096#step:7:18

The exact conditions are fuzzy. Oddly enough gcc 12.3.0 and the SDK
same as above are _compatible_:
https://github.com/curl/curl/actions/runs/9791701214/job/27036037162

Also notice that similar errors can also happen in SecureTransport
builds, due to the SDK headers required.

Ref: https://github.com/curl/curl/pull/14097#issuecomment-2208639046
Ref: https://github.com/curl/curl/pull/14091#issuecomment-2205870854
Cherry-picked from #14097
Closes #14119
2024-07-08 12:03:46 +02:00
Viktor Szakats
8cf02025ab
cmake: feature casing fix and tidy-ups
- fix casing of a feature (`Unicode`) in the feature list.
- sort TLS backends case-insensitively.
- sync feature/protocol list heading with `curl -V` and autotools.

Closes #14120
2024-07-08 12:03:46 +02:00
Viktor Szakats
0b81eccd22
GHA: ignore FTP test result in Windows jobs
They are flaky.

E.g.:
- old-mingw-w64 7.3.0: 2001, 2039, 2083
- msvc: 1501, 593 (multiple)

Ref: https://github.com/curl/curl/pull/13599#issuecomment-2119372376
Cherry-picked from #14116
2024-07-07 22:40:43 +02:00
Viktor Szakats
f99c08dba4
GHA: improve vcpkg cache, add BoringSSL ECH and LibreSSL MSVC jobs
- cache on a per-package basis.
  Replace manual caching with a built-in solution. It shares cached
  package builds between jobs, e.g. libssh2 only builds once
  per platform (instead of once per job). Individual packages are built
  as needed (not the whole per-job tree). It also fixes the duplicate
  cache entry issues.
  Ref: https://learn.microsoft.com/en-us/vcpkg/consume/binary-caching-github-actions-cache
  Follow-up to e26cbe20cb #13979
  Follow-up to cb22cfca69 #14077

- add BoringSSL job with ECH enabled. The first such job in the curl CI.

- add LibreSSL job.

- use vcpkg pre-installed on the runner image, instead of rolling our
  own. This is quicker, simpler and more robust.
  Follow-up to e26cbe20cb #13979

- show pre-installed vcpkg and ports version.

- drop `gsasl` dependency till it reaches the pre-installed vcpkg ports.

- re-add `find .` to see the binaries generated.

- simplify setting up `PATH`.

- exclude failing tests for any job enabling WinIDN.

- drop collecting and uploading log archives. We already dump CMake
  logs, and our build doesn't use Ninja. Rest of files weren't generated
  by the curl build. We don't aim to debug vcpkg package builds.

Closes #14090
2024-07-07 17:21:35 +02:00
Tal Regev
cb22cfca69
GHA: add MSVC UWP job, expand jobs with more options
- add new dependencies: brotli, libpsl (requires libicu2) and gsasl.

- enable WinIDN in a job. Exclude failing tests.

- add UWP job and fix the build logic to support it.

- increase timeouts to build the new dependencies.

Assisted-by: Viktor Szakats
Closes #14077
2024-07-07 15:27:11 +02:00
Dan Fandrich
d70ec5c650 tests: fix sshd UserKnownHostsFile path for MinGW/Cygwin
This is the same thing as the previous commit fd194f46 but on the next
line.

Follow-up to 70d2fca2

Ref: #10818
2024-07-06 22:07:25 -07:00
Dan Fandrich
b7b413e75c tests: fix sshd IdentityFile path for MinGW/Cygwin
This was missed during some refactoring more than a year ago and is
causing a warning "Use of uninitialized value $path in pattern match".

Follow-up to 70d2fca2

Ref: #10818
Closes #14113
2024-07-06 20:32:11 -07:00
Viktor Szakats
96a1a05f66
build: add Debug, TrackMemory, ECH to feature list
Also:

- remove stray `ECH` and `HTTPSRR` from cmake protocol list.

- stop excluding `Debug` and `TrackMemory` in `test1013.pl`.

- configure: delete `CURL_CHECK_CURLDEBUG` check.
  Ref: 065047dc62
  This check was effectively doing nothing, except disabling
 `--enable-curldebug` in `curl-config` for
 Cygwin/MSYS/cegcc/OS2/AIX targets with c-ares enabled.

Closes #14096
2024-07-07 00:33:21 +02:00
Dan Fandrich
b9fea261e6 CI: bump the libc6 on the linux-old build
This contains some security fixes for nscd.
2024-07-05 20:22:26 -07:00
Viktor Szakats
612e571cd1
reuse: fix typo in comment
Follow-up to 9104bad820 #14107
2024-07-06 02:39:24 +02:00
Dan Fandrich
ebd61fa34e CI: Fix typo in comment 2024-07-05 14:45:41 -07:00
Dan Fandrich
127aa61f06 curl: follow-up to fix categories in --help
The commit 6483813b was missing changes necessitated by 2abfc75 that
causes a crash. Also, use ARRAYSIZE() for cleaner code.

Follow-up to 6483813b

Ref #14055
2024-07-05 14:44:43 -07:00
Dan Fandrich
6483813be2
curl: list categories in --help
This eliminates the need to run an extra help subcommand to get the
possible categories, reducing the friction in getting relevant help. The
help wording was also slightly tweaked for grammatical accuracy.

Closes #14055
2024-07-05 23:09:47 +02:00
Daniel Stenberg
ac7035925a
RELEASE-NOTES: synced 2024-07-05 23:06:24 +02:00
renovate[bot]
4d0b925890
GHA: update actions/upload-artifact and actions/download-artifact
update actions/upload-artifact digest to 0b2256b
update actions/download-artifact digest to fa0a91b

Closes #14111
Closes #14110
2024-07-05 22:59:18 +02:00
Max Mehl
9104bad820
reuse: switch to REUSE 3.2 and REUSE.toml
- remove scripts/copyright.pl

Closes #14107
2024-07-05 20:47:48 +02:00
Yedaya Katsman
0f12ee8b39
curl: move more options to deprecated category
--no-npn, --sslv2, --sslv3

Closes #14109
2024-07-05 20:45:00 +02:00
Stefan Eissing
fe83133d5e
multi: pollset assertion only when IP connected
Give warning for an empty pollset only when the connection has at least
IP connectivity. There are cases where the connect in QUIC makes another
attempt on a timeout and no socket will be available during that.

Closes #14108
2024-07-05 17:00:58 +02:00
Daniel Stenberg
2abfc759b9
cmdline-opts: category cleanup
Option cleanups:

 --get is not upload
 --form* are post
 - added several options into ldap, smtp, imap and pop3
 - shortened the category descriptions in the list

category curl fixes:

 --create-dirs removed from 'curl'
 --ftp-create-dirs removed from 'curl'
 --netrc moved to 'auth' from 'curl'
 --netrc-file moved to 'auth' from 'curl'
 --netrc-optional moved to 'auth' from 'curl'
 --no-buffer moved to 'output' from 'curl'
 --no-clobber removed from 'curl'
 --output removed from 'curl'
 --output-dir removed from 'curl'
 --remove-on-error removed from 'curl'

Add a "global" category:

- Made all "global" options set this category

Add a "deprecated" category:

- Moved the deprecated options to it (maybe they should not be in any
 category long term)

Add a 'timeout' category

- Put a number of appropriate options in it

Add an 'ldap' category

- Put the LDAP related option in there

Remove categories "ECH" and "ipfs"

- They should not be categories. Had only one single option each.

Remove category "misc"

- It should not be a category as it is impossible to know when to browse
  it.

--use-ascii moved to ftp and output
--xattr moved to output
--service-name moved to auth

Managen fixes:

- errors if an option is given a category name that is not already setup
  for in code

- verifies that options set `scope: global` also is put in category
  `global´

Closes #14101
2024-07-05 11:05:50 +02:00
Stefan Eissing
18c61aa036
GHA: configure OpenSSL's libdir as 'lib' only
Also mention in HTTP3.md

OpenSSL has a bug that messes the config `--libdir=path` to become the
wrong path in its pkgconfig files. If we just pass `--libdir=lib` it
should avoid this.

Ref: #14099
See also: https://github.com/openssl/openssl/issues/23569

Closes #14102
2024-07-05 10:16:08 +02:00
Daniel Stenberg
655557f2b3
tool_operate: simplify return code handling from url_proto()
The additional checks were superfluous as it would only ever return
error if one of those protocols were set. Also: a returned error
*should* mean get out of there, without having to check more conditions.

Closes #14104
2024-07-05 08:56:22 +02:00
Daniel Stenberg
bf88cdb790
tool_operate: check for --disable case *sensitively*
curl command line options are specified with the correct casing or they
don't match

Closes #14103
2024-07-05 08:55:40 +02:00
Stefan Eissing
b8e098a057
transfer: avoid polling socket every transfer loop
Improve download performance, minimal effort.

Do not poll the socket for pending data every transfer loop iteration.
This gives 10-20% performance gains on large HTTP/1.1 downloads (on my
machine).

Closes #14098
2024-07-04 23:02:41 +02:00
Viktor Szakats
2583d11f22
tests: delete CharConv remains
Closes #14100
2024-07-04 18:12:40 +02:00
Viktor Szakats
b5d565faa8
GHA: bump macOS CMake job parallelism to 4 (nproc+1) [ci skip]
To match autotools ones and the rest of workflows.

Follow-up to 464282ddfb #13807
2024-07-04 18:04:02 +02:00
Yedaya Katsman
62bfcb0c44
help: add flags to output and ssh categories
- Add --output, --remove-on-error, --output-dir and --created-dirs to
  the output help category

- Add --hostpubmd5, --hostpubsha256, --insecure (-k), and --pubkey to
  the ssh help category

Closes #14076
2024-07-04 09:58:23 +02:00
Stefan Eissing
82b32b344e
TODO: remove item about 'SSL_peak'
GnuTLS todo item about using an equivalent of `SSL_peak()`, which
nicely escaped the word checks, is no longer relevant.

We do not use `SSL_peek()` anymore since connection filters were
introduced.

Closes #14091
2024-07-04 09:56:32 +02:00
renovate[bot]
465985291f
GHA: update dependency gnutls/gnutls to v3.8.6
Closes #14094
2024-07-04 09:55:41 +02:00
renovate[bot]
105be9dbc9
GHA: update fsfe/reuse-action action to v4
Closes #14095
2024-07-04 09:53:48 +02:00
Viktor Szakats
fa2c575617
GHA: Windows job exclusions tweaks
- disable SMTP tests in MSYS2/mingw-w64 and MSVC jobs.
  On the suspicion of sometimes hanging:
  https://github.com/curl/curl/actions/runs/9346162475/job/25720437944?pr=13855#step:14:2838
  https://github.com/curl/curl/actions/runs/9758011305/job/26931678639?pr=14084#step:14:2834
  https://github.com/curl/curl/actions/runs/9774468536/job/26982805294#step:11:4731

- run TFTP, MQTT, WebSockets tests in MSYS2/msys jobs again.

- switch hanging old-mingw-w64 7.3.0 job to Release (from Debug).
  Guessing here, 9.5.0 is more solid, and one difference is
  Debug/Release mode. Let's match 7.3.0 with that and see how it changes
  hangs and flakiness.
  The other difference is Unicode ON in 7.3.0. Flaky 6.3.0 was also
  Debug, with Unicode OFF:
  217878bade #13566.
  (Unicode unlikely to play a role here IMO.)
  If 7.3.0 keeps hanging / remains flaky I'll consider disabling its
  test runs.

- opt-out from vcpkg telemetry.

Ref: https://github.com/curl/curl/pull/13599#issuecomment-2119372376
Closes #14085
2024-07-03 23:54:56 +02:00
renovate[bot]
d4eeb3dd6c
Dockerfile: update debian:bookworm-slim to 39868a6
Closes #14083
2024-07-03 08:39:27 +02:00
Daniel Stenberg
38bdef0220
FEATURES.md: refresh
- added lots of missing stuff
- rearranged a little
- remove all footnotes

Closes #14086
2024-07-03 08:35:56 +02:00
Daniel Stenberg
9f968478f9
RELEASE-NOTES: synced 2024-07-03 00:27:32 +02:00
Daniel Stenberg
bf9b22a6f7
curl_easy_perform.md: call it network transfer, not file transfer 2024-07-02 22:55:04 +02:00
Viktor Szakats
092c07d119
winbuild: MS-DOS batch tidy-ups
- prefer `.bat` extension over `.cmd` for MS-DOS batch, which also
  avoids confusion with OS/400 `.cmd` files.
- cleanup `echo` quotes, drop them consistently.
- delete empty output line from one of the error branches.
- prefer lowercase commands like the rest of MS-DOS batches.
- delete a contraction.
- drop backticks from error message.
- use `nmake.exe` consistently.
- use equal/not-equal operator style consistently.
- inline a single-line `if` branch.
- delete exceptions and rules dealing with Windows `.cmd` extension.

Closes #14084
2024-07-02 19:26:15 +02:00
Stefan Eissing
480883cf27
multi: fix pollset during RESOLVING phase
- add a DEBUGASSERT for when a transfer's pollset should not be empty.
- move write unpausing from transfer loop into curl_easy_pause. This
  make sure that the url_updatesocket() finds the correct state when
  updating socket events.
- fix HTTP/2 proxy during connect phase to set sockets correctly
- fix test2600 to simulate a socket set
- move write unpausing from transfer loop into curl_easy_pause. This
  make sure that the url_updatesocket() finds the correct state when
  updating socket events.
- waiting for the resolver to deliver might not involve any sockets to
  wait for. Do not generate a warning.

Fixes #14047
Closes #14074
2024-07-02 11:17:38 +02:00
Daniel Stenberg
75763a3e88
cmdline-opts: shorten six help texts
o --location-trusted
 o --next
 o --parallel-immmediate
 o --pinnedpubkey
 o --proxy-pass
 o --proxy-ssl-allow-beast

Closes #14075
2024-07-02 07:51:57 +02:00
Daniel Stenberg
9784f9acf3
managen: fix removing backticks from subtitles
It erroneously removed them from the wrong variable.

Closes #14081
2024-07-02 07:47:49 +02:00
Viktor Szakats
80769b9c9e
cmake: show protocols, then features
To match the order used by `curl -V` and `./configure`.

Closes #14082
2024-07-02 01:03:09 +02:00
Viktor Szakats
9484beaa49
cmdline-docs: fix --proxy-ca-native example + tidy-ups
Also:
- fix an indentation.
- fix capitalized option in comment.

Closes #14078
2024-07-02 00:09:13 +02:00
Viktor Szakats
6b10edb764
cmake: sync protocol/feature list with curl -V output
- sort features case-insensitively.
  Requires CMake v3.13.0.
  Follow-up to 0f26abeef1 #14063

- convert protocol list to lowercase.
  But leave it uppercase in `curl-config`.

Closes #14066
2024-07-01 23:24:46 +02:00
Viktor Szakats
a3e613a84f
GHA/badwords.yml: fixup indent for yamllint [ci skip] 2024-07-01 23:24:11 +02:00
renovate[bot]
e0b1ed9098
GHA: update dependency awslabs/aws-lc to v1.31.0
Closes #14080
2024-07-01 23:10:41 +02:00
Daniel Stenberg
ca6b05b6dc
GHA/badwords.yml: check source code wording
Closes #14073
2024-07-01 22:59:19 +02:00
Daniel Stenberg
c074ba64a8
code: language cleanup in comments
Based on the standards and guidelines we use for our documentation.

 - expand contractions (they're => they are etc)
 - host name = > hostname
 - file name => filename
 - user name = username
 - man page => manpage
 - run-time => runtime
 - set-up => setup
 - back-end => backend
 - a HTTP => an HTTP
 - Two spaces after a period => one space after period

Closes #14073
2024-07-01 22:58:55 +02:00