Commit Graph

32870 Commits

Author SHA1 Message Date
Viktor Szakats
cd51bb503a
cmake: add debug function to dump all variables
Place a `curl_dumpvars()` call anywhere in the CMake code to use it.

Closes #14439
2024-08-07 15:10:52 +02:00
Viktor Szakats
e20413980c
GHA/macos: tweak toolchain dump steps
- use documented flavour of `xcrun` option.

- show SDK version with a dedicated command.
  (Sometimes the SDK path is a symlink and doesn't tell the version.
  This is not at the moment the case in CI, but handle it anyway.)

- align group header with reality.
  Preinstalled vs. installed Homebrew packages can be recognized
  by their directory timestamps. Installed ones have a current date.

Closes #14434
2024-08-07 15:10:51 +02:00
Bo Anderson
588a6e334a
idn: more strictly check AppleIDN errors
UIDNA API returns two error values but we were only checking one.
Checking both better aligns the behaviour with that of libidn2.

Closes #14431
2024-08-07 14:59:07 +02:00
Bo Anderson
a35687831f
idn: support non-UTF-8 input under AppleIDN
This aligns the behaviour with libidn2 and the curl documentation.

Closes #14431
2024-08-07 14:59:01 +02:00
Jiacai Liu
07843d8167
BINDINGS: add zig binding
Closes #14437
2024-08-07 14:51:09 +02:00
Viktor Szakats
493c6d79eb
cmake: delete MSVC warning suppression for tests/server
Server code no longer produces this warning.

Closes #14428
2024-08-07 11:35:16 +02:00
Viktor Szakats
b1153820f4
dist: add missing test_*.py scripts
Closes #14427
2024-08-07 11:35:16 +02:00
Dan Fandrich
5f9426ec43 tests: show snapshot commit in testcurl
This disambiguates the source code being tested. The output format is
the same as when testing out of a git repo, but with no description and
a long hash.

Ref: #14363
Closes #14429
2024-08-07 00:22:38 -07:00
Pete Cordell
0011df47bc
ws: flags to opcodes should ignore CURLWS_CONT flag
When converting WebSocket flags such as CURLWS_TEXT | CURLWS_CONT we
want to exclude CURLWS_CONT from the lookup.

Closes #14397
2024-08-07 09:06:30 +02:00
Stefan Eissing
b102763c19
curl: fix --test-event --parallel
(in debug-builds)

Fix implementation in curl using libuv to process parallel transfers.
Add pytest capabilities to run test cases with --test-event.

- fix uv_timer handling to carry correct 'data' pointing to uv context.
- fix uv_loop handling to reap and add transfers when possible
- fix return code when a transfer errored

Closes #14413
2024-08-07 08:57:05 +02:00
Jan Venekamp
1b25448760
curl: warn on unsupported SSL options
Closes #14406
2024-08-07 08:50:53 +02:00
Jan Venekamp
5c2ab55abe
vtls: add SSLSUPP_CIPHER_LIST
Added SSLSUPP_CIPHER_LIST so be able to differniate SSL Backends
that support CURLOPT_SSL_CIPHER_LIST.

Closes #14406
2024-08-07 08:50:42 +02:00
Daniel Stenberg
cd4aee156f
tests: ignore QUIT from FTP protocol comparisons
- QUIT is not an important FTP command
- curl only sends it "best effort", meaning it might not be sent
- it is a known "flaky" thing in test output because of this

Closes #14404
2024-08-07 08:48:56 +02:00
Daniel Stenberg
b3490c5bc4
RELEASE-NOTES: synced 2024-08-07 08:25:43 +02:00
Stefan Eissing
06c5829dab
curl: support repeated use of the verbose option; -vv etc
- make mentioning `-v` on the curl command line increase the
  verbosity of the trace output
- related discussion https://github.com/curl/curl/discussions/13810
- make a single -v revert all previous -v+ changes
- make --no-verbose also reset all trace configs

Closes #13977
2024-08-07 08:14:36 +02:00
Daniel Stenberg
53146dd262
tool_help: handle longer lines, exit on too long
Follow-up to 9a0cf56471

- increase the buffer to handle 160 characters manpage lines
- add another assert
- if the line buffer gets full, abort

Ideally, we add another step in the build process that makes the build
fail if this long lines are used.

Closes #14422
2024-08-07 00:31:07 +02:00
Daniel Stenberg
48818a41af
tests/runner: only allow [!A-Za-z0-9_-] in %if feature names
... in the %if preprocessor parser.

This guards against accidentally getting a carriage-return part of the
feature name.

Fixes #14403
Closes #14411
2024-08-07 00:30:17 +02:00
Daniel Stenberg
b0394b1535
runtests: if DISABLED cannot be read, error out
Because then runtests cannot disable the right set of tests and things
will be annoying.

Closes #14411
2024-08-07 00:30:17 +02:00
Viktor Szakats
c6fb9895b6
cmake: cleanup header paths
- sync build-dir/source-dir header path order with autotools, by
  including build-dir first, then source-dir.
  This prevents out-of-tree builds breaking due to leftover generated
  headers in the source tree.

- tests/unit: move `src` ahead of `libtest` in header path, syncing with
  autotools.

- stop adding non-existing generated `include` dir to header path.
  There are no generated `include` headers and this directory is either
  missing in out-of-tree builds or the same as the one already added
  globally via the root `CMakeLists.txt`.

- lib: stop adding a duplicate source include directory to the header
  path.
  It's already added globally via the root `CMakeLists.txt`.

- lib: stop adding the project root to the header path.

- docs/examples: drop internal header paths.
  Examples do not and should not use internal headers.

- replace `curl_setup_once.h` in comments with `curl_setup.h`,
  the header actually used, and also referred to in autotools comments.

- add comment why we need `src` in include path for `tests/server`.

- add quotes around header directories.

Closes #14416
2024-08-06 22:33:05 +02:00
Viktor Szakats
ada8ebe18c
GHA/macos: enable AppleIDN in autotools job
- make `--with-apple-idn` override libidn2, in sync with cmake and
  `lib/curl_setup.h`.

- sync detection function name with cmake.

- limit AppleIDN feature check to Darwin.
  (also drop !WinIDN precondition check.)

Follow-up to 8de8fe8c98 #14401
Closes #14419
2024-08-06 22:33:05 +02:00
Viktor Szakats
7b1c0ab75e
Makefile.mk: fixup enabling libidn2
Replicate the method used by autotools and cmake to enable libidn2.
This way `lib/curl_setup.h` sets `USE_LIBIDN2` automatically.

Before this patch, `USE_LIBIDN2` was enabled directly, shortcutting
internal logic prioritizing IDN backends.

(This is academic now because `Makefile.mk` no longer supports
other IDN backends. But still useful for clarity.)

Closes #14421
2024-08-06 22:33:05 +02:00
Viktor Szakats
ea3dfcb361
cmake: drop unused HAVE_IDNA_STRERROR
Unused since 9c91ec7781

Closes #14420
2024-08-06 22:33:05 +02:00
Viktor Szakats
6712bd6006
cmake: show CMake platform/compiler flags
To better understand which is active in which build job.

Closes #14417
2024-08-06 22:33:04 +02:00
Daniel Stenberg
dcc5209520
GHA: run badwords check on tests/*.md too
With several wording updates accordingly.

Closes #14412
2024-08-06 13:43:25 +02:00
Daniel Stenberg
91fcbc5d1a
dist: drop buildconf
The documented and mandated step has been to not use buildconf but to
invoke 'autoreconf -fi' for four years already.

This change only drops buildconf from the release tarball, it remains
present in git for now.

Follow-up to 85868537d6

Closes #14412
2024-08-06 13:43:20 +02:00
Tal Regev
8577f4ca08
cmake: add math library when using wolfssl and ngtcp2
Closes #14343
2024-08-06 11:48:21 +02:00
Yedaya Katsman
bfa939d062
docs: mention "@-" in more places
Closes #14402
2024-08-06 11:47:27 +02:00
Viktor Szakats
67d5e3624b
cmake: replace an MSVC_VERSION with MSVC
Where the actual version is not relevant.

Follow-up to ce81aeb877
Closes #14410
2024-08-06 11:05:49 +02:00
Viktor Szakats
72ae0d86a4
cmake: use numeric comparison for HAVE_WIN32_WINNT
Turns out CMake supports numeric comparison with hexadecimal values.
Confirmed in GHA/linux-old with CMake 3.7.2. I could not find
documentation about this, but our CMakeLists.txt already used it before
this patch.

Extend that method to two more comparisons.

Also pad the value in the existing one to 4 digits.

The padding/lowercasing logic when setting `HAVE_WIN32_WINNT` is no
longer required, but keep it anyway for uniform log output.

Follow-up to 2100d9fde2 #12044
Closes #14409
2024-08-06 11:05:48 +02:00
Daniel Stenberg
8de8fe8c98
configure: detect AppleIDN
Follow-up to add22feeef

Closes #14401
2024-08-06 09:36:07 +02:00
Viktor Szakats
232302f88a
cmake: add Linux CI job, fix pytest with cmake
- extend existing Linux workflow with CMake support.
  Including running pytest the first time with CMake.

- cmake: generate `tests/config` and `tests/http/config.ini`.
  Required for pytest tests.
  Uses basic detection logic. Feel free to take it from here.
  Also dump config files in a CI step for debugging purposes.

- cmake: build `tests/http/clients` programs.

- fix portability issues with `tests/http/clients` programs.
  Some of them use `getopt()`, which is not supported by MSVC.
  Fix the rest to compile in CI (old-mingw-w64, MSVC, Windows).

- GHA/linux: add CMake job matching an existing autotools one.

- GHA/linux: test `-DCURL_LIBCURL_VERSIONED_SYMBOLS=ON`
  in the new CMake job.

- reorder testdeps to build server, client tests first and then
  libtests and units, to catch errors in the more complex/unique
  sources earlier.

- sort list in `tests/http/clients/Makefile.inc`.

Closes #14382
2024-08-06 02:40:32 +02:00
Viktor Szakats
f7d5f47059
cmake: add support for CURL_USE_LIBUV option
Also use an `#undef` hack for CMake Unity builds to avoid the previously
included `memdebug.h` header messing up the declarations pulled in by
`uv.h`:
```
In file included from ~/curl/bld/src/CMakeFiles/curl.dir/Unity/unity_0_c.c:88:
In file included from ~/curl/src/tool_operate.c:54:
In file included from /usr/local/Cellar/libuv/1.48.0/include/uv.h:71:
In file included from /usr/local/Cellar/libuv/1.48.0/include/uv/unix.h:34:
/Library/Developer/CommandLineTools/SDKs/MacOSX13.1.sdk/usr/include/netdb.h:269:7: error: expected parameter declarator
void            freeaddrinfo(struct addrinfo *);
                ^
~/curl/lib/memdebug.h:167:31: note: expanded from macro 'freeaddrinfo'
  curl_dbg_freeaddrinfo(data, __LINE__, __FILE__)
                              ^
```

Follow-up to 38d334e3e1 #14298
Closes #14399
2024-08-06 02:40:16 +02:00
dependabot[bot]
e64e62cc7d
GHA/windows: bump msys2/setup-msys2 from 2.24.0 to 2.24.1
Bumps [msys2/setup-msys2](https://github.com/msys2/setup-msys2) from 2.24.0 to 2.24.1.
- [Release notes](https://github.com/msys2/setup-msys2/releases)
- [Changelog](https://github.com/msys2/setup-msys2/blob/main/CHANGELOG.md)
- [Commits](5df0ca6cbf...ddf331adae)

---
updated-dependencies:
- dependency-name: msys2/setup-msys2
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <support@github.com>
Closes #14396
2024-08-05 23:35:18 +02:00
Austin Moore
cf3e3d93d1
aws_sigv4: fix canon order for headers with same prefix
If a request containing two headers that have equivalent prefixes (ex.
"x-amz-meta-test:test" and "x-amz-meta-test-two:test2") AWS expects the
header with the shorter name to come first. The previous implementation
used `strcmp` on the full header. Using the example, this would result
in a comparison between the ':' and '-' chars and sort
"x-amz-meta-test-two" before "x-amz-meta-test", which produces a
different "StringToSign" than the one calculated by AWS.

Test 1976 verifies

Closes #14370
2024-08-05 23:32:10 +02:00
Jan Venekamp
f3e07e5c55
docs: wolfssl and mbedtls add CURLOPT_TLS13_CIPHERS support
Documentation for #14384 and #14385

Closes #14386
2024-08-05 23:29:11 +02:00
Jan Venekamp
4c12892411
wolfssl: add CURLOPT_TLS13_CIPHERS support
Bring setting ciphers with WolfSSL in line with other SSL backends,
to make the curl interface more consistent across the backends.

Now the tls1.3 ciphers are set with the --tls13-ciphers option, when
not set the default tls1.3 ciphers are used. The tls1.2 (1.1, 1.0)
ciphers are set with the --ciphers option, when not set the default
tls1.2 ciphers are used. The ciphers available for the connection
are now a union of the tls1.3 and tls1.2 ciphers.

This changes the behaviour for WolfSSL when --ciphers is set, but
--tls13-ciphers is not set. Now the ciphers set with --ciphers
are combined with the default tls1.3 ciphers, whereas before solely
the ciphers of --ciphers were used.

Thus before when no tls1.3 ciphers were specified in --ciphers,
tls1.3 was completely disabled. This might not be what the user
expected, especially as this does not happen with OpenSSL.

Closes #14385
2024-08-05 23:28:04 +02:00
Daniel Stenberg
a18680f501
VULN-DISCLOSURE-POLICY.md: small typo fix 2024-08-05 17:15:31 +02:00
Viktor Szakats
82bbb386a3
cmake: fix GSS_VERSION for Heimdal found via pkg-config
Previously used source variable was never defined, possibly due
to a copy-paste-edit typo.

Closes #14393
2024-08-05 16:37:19 +02:00
Jan Venekamp
3f7dc8a404
mbedtls: add CURLOPT_TLS13_CIPHERS support
Bring setting ciphers with mbedTLS in line with other SSL backends,
to make the curl interface more consistent across the backends.

Now the tls1.3 ciphers are set with the --tls13-ciphers option, when
not set the default tls1.3 ciphers are used. The tls1.2 (1.1, 1.0)
ciphers are set with the --ciphers option, when not set the default
tls1.2 ciphers are used. The ciphers available for the connection
are now a union of the tls1.3 and tls1.2 ciphers.

This changes the behaviour for mbedTLS when --ciphers is set, but
--tls13-ciphers is not set. Now the ciphers set with --ciphers
are combined with the default tls1.3 ciphers, whereas before solely
the ciphers of --ciphers were used.

Thus before when no tls1.3 ciphers were specified in --ciphers,
tls1.3 was completely disabled. This might not be what the user
expected, especially as this does not happen with OpenSSL.

Closes #14384
2024-08-05 16:01:20 +02:00
Moritz Buhl
d266d19d86
ngtcp2: use NGHTTP3 prefix instead of NGTCP2 for errors in h3 callbacks
Closes #14394
2024-08-05 15:58:45 +02:00
Daniel Stenberg
b9d465c89f
tool_help: fix a NULL deref in the --help option code
Follow-up to 9a0cf56471

Pointed out by CodeSonar

Closes #14391
2024-08-05 15:55:38 +02:00
Daniel Stenberg
0238a9b0d7
KNOWN_BUGS: "special characers" in URL works with aws-sigv4
Since #13754
Pointed-out-by: Slaven Rezić
Bug: https://github.com/curl/curl/issues/13754#issuecomment-2268374427
Closes #14392
2024-08-05 15:25:47 +02:00
Daniel Stenberg
38d334e3e1
curl: use libuv for parallel transfers with --test-event
add --with-libuv to configure to (optionally) use it in debug-builds to
drive the event-based API

Use curl_multi_socket_action() and friends to drive parallel transfers.

tests/README has brief documentation for this

Closes #14298
2024-08-05 09:12:09 +02:00
Daniel Stenberg
7c31ceb5d0
RELEASE-NOTES: synced 2024-08-05 09:03:11 +02:00
Stefan Eissing
35bf766280
http2: improved upload eos handling
- replace the counting of upload lengths with the new eos send flag
- improve frequency of stream draining to happen less on events where it
  is not needed
- this PR is based on #14220

http2, cf-h2-proxy: fix EAGAINed out buffer
- in adjust pollset and shutdown handling, a non-empty `ctx->outbufq`
  must trigger send polling, irregardless of http/2 flow control
- in http2, fix retry handling of blocked GOAWAY frame

test case improvement:
- let client 'upload-pausing' handle http versions

Closes #14253
2024-08-05 08:53:55 +02:00
Stefan Eissing
344ba8c883
wolfssl: improve shutdown handling
Improve handling of shutdown when sending gets blocked.

Add workaround for <https://github.com/wolfSSL/wolfssl/issues/7784>
where wolfSSL keeps on adding close notify messages to its outgoing
buffer on ever attempt.

Closes #14376
2024-08-05 08:49:20 +02:00
Stefan Eissing
4494005b50
openssl: improve shutdown handling
Make sure that `io_need` is cleared and set at the filter operations.
Add some more tracing for shutdown situations.

Improve shutdown handling for blocked sends. OpenSSL is a bit tricksy
here that it only reports WANT_WRITE on SSL_shutdown(), but never on
SSL_read() on blocked sends. So we need to use both.

At last, set SSL_MODE_ACCEPT_MOVING_WRITE_BUFFER when available since we
are not always retrying sends from the very same address, as testing
showed.

Closes #14375
2024-08-05 08:48:06 +02:00
Stefan Eissing
6f19210667
bearssl: improve shutdown handling
- replace own "adjust_pollset()" with vtls' standard one
- set io_need always accordingly
- handle EAGAIN on shutdown

Closes #14374
2024-08-05 08:46:24 +02:00
Daniel Stenberg
ed2850456c
configure: fail if PSL is not disabled but not found
Regression since 9b3f67e (shipped in 8.7.0)
Reported-by: Ryan Carsten Schmidt
Fixes #14373
Assisted-by: Viktor Szakats
Closes #14379
2024-08-05 08:33:58 +02:00
Daniel Stenberg
7d45b52166
KNOWN_BUGS: mention AppleIDN and WinIDN test problems
Closes #14176
Closes #14387
2024-08-05 08:23:37 +02:00