Commit Graph

32072 Commits

Author SHA1 Message Date
Dan Fandrich
5b94eced85 tests: Fix uninitialized value warning
The check for an option must be predicated on options existing at all.

Follow-up to f7cc9e91
2024-04-16 19:53:22 -07:00
MonkeybreadSoftware
add22feeef
idn: add native AppleIDN (icucore) support for macOS/iOS
I implemented the IDN functions for macOS and iOS using Unicode
libraries coming with macOS and iOS.

Builds and runs here on macOS 14.2.1. Also verified to load and
run on older macOS version 10.13.

Build requires macOS SDK 13 or equivalent.

Set `-DUSE_APPLE_IDN=ON` CMake option to enable it.
With autotools and other build tools, set these manual options:
```
CPPFLAGS=-DUSE_APPLE_IDN
LIBS=-licucore
```

Completes TODO 1.6.

TODO: add autotools option and feature-detection.

Refs: #5330 #5371
Co-authored-by: Viktor Szakats
Closes #13246
2024-04-17 00:24:09 +02:00
Stefan Eissing
08d10d2a00
http3: extend download abort tests, fixes in ngtcp2
- fix flow handling in ngtcp2 to ACK data on streams
  we abort ourself.
- extend test_02_23* cases to also run for h3
- skip test_02_23* for OpenSSL QUIC as it gets stalled
  on progressing the connection

Closes #13374
2024-04-16 23:48:22 +02:00
Daniel Stenberg
f7cc9e9177
tests: add -q as first option when invoking curl for tests
To reduce the risk that the user running the tests has a .curlrc present
that messes things up.

Support 'option="no-q"' for the <command> tag to switch it off on demand.
Use this new feature in test 433 and 436.

Ref: #13284
Closes #13387
2024-04-16 23:40:23 +02:00
Daniel Stenberg
d28f74913c
dmaketgz: release tarball generation using docker
For easier reproducibility.

Mention using this script in RELEASE-PROCEDURE

Closes #13388
2024-04-16 23:38:55 +02:00
Viktor Szakats
dad126b840
cmake: update ECH code and minor fixups
- `openssl_check_symbol_exists()` expects a 4th argument now.
  Follow-up to edc2702a1f #13373

- minor comment/script touch-ups.
  Follow-up to a362962b72 #11922

- fix indentation.

Closes #13383
2024-04-16 19:47:13 +02:00
Viktor Szakats
57af812e5f
tests: fix shellcheck issues in ech_tests.sh
Add double-quotes where missing.

Follow-up to a362962b72 #11922
Closes #13382
2024-04-16 19:47:13 +02:00
Viktor Szakats
5d3016adf7
dist: add ECH files to tarball
Also sort `EXTRA_DIST` list in `tests/Makefile.am` and make it diffable.

Follow-up to a362962b72 #11922
Closes #13381
2024-04-16 19:47:13 +02:00
Viktor Szakats
09f0390e11
openvms: look for USE_IPV6 in config.h (was: ENABLE_IPV6)
The OpenVMS script `config_h.com` is parsing the config header
generated by autotools. Let's make it look for the macro name we now
use universally across the codebase.

Follow-up to e411c98f70 #13349
Closes #13360
2024-04-16 19:38:47 +02:00
daniel-j-h
41c03b4c98
Dockerfile: for release automation and reproducibility
Closes #13250
2024-04-16 16:53:25 +02:00
Stefan Eissing
270a25c011
cw-out: improved error handling
- remember error encountered in invoking write callback and always fail
  afterwards without further invokes

- check behaviour in test_02_17 with h2-pausing client

Reported-by: Pavel Kropachev
Fixes #13337
Closes #13340
2024-04-16 15:52:10 +02:00
Daniel Stenberg
5e3fd347c5
version: add "ECH" as a feature
If available

Follow-up to a362962b7
Closes #13378
2024-04-16 13:24:08 +02:00
Daniel Stenberg
a1ecd0ba6b
CURLOPT_ECH: polish
- remove the pointer to build instructions, it won't work in manpages
- add see-also
- minor white space edits

Closes #13379
2024-04-16 09:58:45 +02:00
Viktor Szakats
f81f60206d
tidy-up: whitespace [ci skip] 2024-04-16 09:53:39 +02:00
Viktor Szakats
3774b8a9a7
mbedtls: fix building with v3 in CMake Unity mode
Before this patch the internal feature detection macro
`HAS_MBEDTLS_RESULT_CODE_BASED_FUNCTIONS` was defined in three files,
with an incomplete logic in one of them. In Unity mode that spilled
into another source file and broke the build.

Closes #13377
2024-04-16 09:36:43 +02:00
Viktor Szakats
edc2702a1f
cmake: add librtmp/rtmpdump option and detection
Add CMake option `USE_LIBRTMP`. Disabled by default.

This library requires OpenSSL TLS-backend when linked statically.

Follow-up to 6eb9e65781 #13364
Closes #13373
2024-04-16 09:36:42 +02:00
Stephen Farrell
a362962b72
TLS: add support for ECH (Encrypted Client Hello)
An EXPERIMENTAL feature used with CURLOPT_ECH and --ech.

Closes #11922
2024-04-16 08:10:53 +02:00
Daniel Stenberg
565d28dc8e
RELEASE-NOTES: synced 2024-04-15 23:47:08 +02:00
Daniel Stenberg
be659030ba
multi: introduce SETUP state for better timeouts
Since we can go to the CONNECT state from PENDING, potentially multiple
times for a single transfer, this change introdues a SETUP state that
happens before CONNECT when doing a new transfer.

Now, doing a redirect on a handle goes back to SETUP (not CONNECT like
before) and we initilize the connect timeout etc in SETUP. Previously,
we would do it in CONNECT but that would make it unreliable in cases
where a transfer goes in and out between CONNECT and PENDING multiple
times.

SETUP is transient, so the handle never actually stays in that state.

Additionally: take care of timeouts of PENDING transfers in
curl_multi_perform()

Ref: #13227
Closes #13371
2024-04-15 23:42:06 +02:00
Tal Regev
6eb9e65781
cmake: forward USE_LIBRTMP option to C
Define in C `USE_LIBRTMP` if user requested it from cmake.

Closes #13364
2024-04-15 23:32:23 +02:00
Daniel Stenberg
dde4b3855e
curl_version_info: provide librtmp version
Ref: https://github.com/curl/curl/pull/13364#issuecomment-2054151942
Reported-by: talregev on github
Closes #13368
2024-04-15 16:48:34 +02:00
blankie
e1f1ec028a
docs: clarify CURLOPT_MAXFILESIZE and CURLOPT_MAXFILESIZE_LARGE
The bounds of the size parameter were not specified, and nor was it
specified how to disable the maximum file size check.

The documentation also incorrectly stated that CURLOPT_MAXFILESIZE
always returns CURLE_OK and that CURLOPT_MAXFILESIZE_LARGE only returns
CURLE_OK or CURLE_UNKNOWN_OPTION.

It also did not mention what the default value is, which is zero. This
commit updates the documentation to make note of all these things.

Closes #13372
2024-04-15 13:38:27 +02:00
Patrick Monnerat
5e46c2909f
OS400: post-shellcheck changes adjustments
Build scripts must be executed by the os/400 shell (sh), not bash which
is a PASE program.

Shell function get_make_vars() escaping reworked to match $() subcommand
construct.

Follow-up to 8a622baf9e
Closes #13366
2024-04-15 08:58:13 +00:00
Viktor Szakats
8bac53ecf0
OS400: tidy-up
Drop/fixup mods trying to make some syntax highlighters happier.

Follow-up to 8a622baf9e #13309
Closes #13362
2024-04-15 08:58:09 +00:00
Daniel Stenberg
a1ec035afc
multi: timeout handles even without connection
When there is a "change" in a multi handle and pending handles are moved
back to the main list to be retested if they can proceed further (for
example a previous transfer completed or a connection has a confirmed
multiplexed state), the timeout check in multi_runsingle() would not
trigger because it required an established connection.

This could make a pending tranfer go back to pending state even though
it had been "in progress" for a longer time than permitted. By removing
the requirement for an associated connection, the timeout check will be
done proper even for transfers that has not yet been assigned one.

Ref #13227
Reported-by: Rahul Krishna M
Closes #13276
2024-04-15 09:49:14 +02:00
Patrick Monnerat
6e8a6039b8
mprintf: check fputc error rather than matching returned character
OS/400 ascii fputc wrapper deviates from the posix standard by the
fact that it returns the ebcdic encoding of the original ascii
character. Testing for a matching value for success will then always
fail.

This commit replaces the chariacter comparison by an explicit error
return check.

Follow-up to ef2cf58
Closes #13367
2024-04-15 08:38:14 +02:00
Viktor Szakats
0469c68f49
ci: add CMake build variation, fixup libssh detection in linux-old
To test without c-ares and hit `easy_lock.h` on an old system. Use this
new build step to introduce small variations, and also test libssh2.

Also add workaround to existing job to enable libssh. (CMake's generic
auto-detection doesn't seem to work here.):
```
CMake Warning at CMakeLists.txt:908 (find_package):
  Could not find a package configuration file provided by "libssh" with any
  of the following names:

    libsshConfig.cmake
    libssh-config.cmake
```
Ref: https://github.com/curl/curl/actions/runs/8661316091/job/23750974358#step:5:69

Closes #13361
2024-04-14 02:30:40 +00:00
Viktor Szakats
49f83c30e4
lib: merge ENABLE_QUIC C macro into USE_HTTP3
Before this patch `lib/curl_setup.h` defined these two macros right
next to each other, then the source code used them interchangeably.

After this patch, `USE_HTTP3` guards all HTTP/3 / QUIC features.
(Like `USE_HTTP2` does for HTTP/2.) `ENABLE_QUIC` is no longer used.

This patch doesn't change the way HTTP/3 is enabled via autotools
or CMake. Builders who enabled HTTP/3 manually by defining both of
these macros via `CPPFLAGS` can now delete `-DENABLE_QUIC`.

Closes #13352
2024-04-13 08:33:27 +00:00
Viktor Szakats
e411c98f70
build: prefer USE_IPV6 macro internally (was: ENABLE_IPV6)
Before this patch, two macros were used to guard IPv6 features in curl
sources: `ENABLE_IPV6` and `USE_IPV6`. This patch makes the source use
the latter for consistency with other similar switches.

`-DENABLE_IPV6` remains accepted for compatibility as a synonym for
`-DUSE_IPV6`, when passed to the compiler.

`ENABLE_IPV6` also remains the name of the CMake and `Makefile.vc`
options to control this feature.

Closes #13349
2024-04-13 08:33:26 +00:00
Dan Fandrich
de66e8ad38 DISTROS: mark rolling release distros
These are ones that are unlikely to have back-ported curl patches.

Closes #13353
2024-04-12 11:28:24 -07:00
Daniel Stenberg
68ce971c1d
mbedtls: cut off trailing newlines from debug logs
To avoid double newlines in the output.

Reported-by: Gisle Vanem
Fixes #13321
Closes #13356
2024-04-12 18:27:45 +02:00
Daniel Stenberg
16bad89239
RELEASE-NOTES: synced 2024-04-12 13:25:44 +02:00
Stefan Eissing
8482ce53dd
CURLINFO_REQUEST_SIZE: fixed, add tests for transfer infos reported
- tests for 'size_request' and other stats reported, for
  presence and consistency

Reported-by: Jonatan Vela
Fixes #13269
Closes #13275
2024-04-12 13:20:09 +02:00
Viktor Szakats
0a4419ae4c
dist: add files missing from release tarball
Closes #13346
2024-04-11 15:52:08 +00:00
Viktor Szakats
bba4c313bc
ci: parallelize more, tidy up cmake commands (distcheck, macos)
Also enable `-DCURL_WERROR=ON` in the Linux cmake build test.

Closes #13343
2024-04-11 15:52:07 +00:00
Toon Claes
0f7be5a519
docs: add CURLOPT_NOPROGRESS to CURLOPT_XFERINFOFUNCTION example
It's important to set `CURLOPT_NOPROGRESS` to `0` if you want your
transfer callback function, set by `CURLOPT_XFERINFOFUNCTION`, getting
called. To emphasize this to the users, add this to the code example.

Closes #13348
2024-04-11 15:59:53 +02:00
RainRat
1087937992
misc: fix typos
Closes #13344
2024-04-11 15:44:22 +02:00
Colin Leroy-Mira
bfe54b0e88
file: add support for getting basic directory listings
Not supported on Windows (yet)

Closes #13137
2024-04-11 12:37:12 +02:00
Viktor Szakats
e14daeb8a4
ci: add curl-for-win builds: Linux MUSL, macOS, Windows
Linux MUSL (llvm/clang), macOS Apple clang, Windows (llvm/clang).

Configured with HTTP/2 and HTTP/3 and other dependencies (the default
curl-for-win) for a comprehensive build test.

```
curl 8.8.0-DEV (x86_64-unknown-linux-musl) libcurl/8.8.0-DEV LibreSSL/3.9.1 zlib/1.3.1 brotli/1.1.0 zstd/1.5.6 libpsl/0.21.5 libssh2/1.11.0 nghttp2/1.61.0 ngtcp2/1.4.0 nghttp3/1.2.0
Protocols: dict file ftp ftps gopher gophers http https imap imaps ipfs ipns mqtt pop3 pop3s rtsp scp sftp smb smbs smtp smtps telnet tftp ws wss
Features: alt-svc AsynchDNS brotli HSTS HTTP2 HTTP3 HTTPS-proxy IPv6 Largefile libz NTLM PSL SSL threadsafe UnixSockets zstd

curl 8.8.0-DEV (x86_64-apple-darwin) libcurl/8.8.0-DEV LibreSSL/3.9.1 zlib/1.3.1 brotli/1.1.0 zstd/1.5.6 libpsl/0.21.5 libssh2/1.11.0 nghttp2/1.61.0 ngtcp2/1.4.0 nghttp3/1.2.0
Protocols: dict file ftp ftps gopher gophers http https imap imaps ipfs ipns ldap ldaps mqtt pop3 pop3s rtsp scp sftp smb smbs smtp smtps telnet tftp ws wss
Features: alt-svc AsynchDNS brotli HSTS HTTP2 HTTP3 HTTPS-proxy IPv6 Largefile libz NTLM PSL SSL threadsafe UnixSockets zstd

curl 8.8.0-DEV (x86_64-w64-mingw32) libcurl/8.8.0-DEV LibreSSL/3.9.1 zlib/1.3.1 brotli/1.1.0 zstd/1.5.6 WinIDN libpsl/0.21.5 libssh2/1.11.0 nghttp2/1.61.0 ngtcp2/1.4.0 nghttp3/1.2.0
Protocols: dict file ftp ftps gopher gophers http https imap imaps ipfs ipns ldap ldaps mqtt pop3 pop3s rtsp scp sftp smb smbs smtp smtps telnet tftp ws wss
Features: alt-svc AsynchDNS brotli HSTS HTTP2 HTTP3 HTTPS-proxy IDN IPv6 Kerberos Largefile libz NTLM PSL SPNEGO SSL SSPI threadsafe UnixSockets zstd
```

Limited to x64, because for build testing the additional CPUs don't add
much value compared to the extra build time. They can be enabled easily
if deemed useful.

To the extent of curl-for-win configuration options, it's trivial to add
further build combinations.

Closes #13335
2024-04-11 07:48:52 +00:00
Viktor Szakats
8a622baf9e
OS400: fix shellcheck warnings in scripts
- use `$()` instead of backticks, and re-arrange double-quotes inside.
- add missing `|| exit 1` to `cd` calls. (could be dropped by using `set -eu`.)
- add `-n` to a few `if`s.
- shorten redirections by using `{} >` (as shellcheck recommended).
- silence warnings where variables were detected as unused (SC2034).
- a couple misc updates to silence warnings.
- switch to bash shebang for `-ot` feature.
- split two lines to unbreak syntax highlighting in my editor. (`$(expr \`, `$(dirname \`)

Also enable CI checks for OS/400 shell scripts.

Ref: #13307
Closes #13309
2024-04-11 07:48:51 +00:00
Stefan Eissing
8dd81bd5db
lib: add Curl_xfer_write_resp_hd
Add method in protocol handlers to allow writing of a single,
0-terminated header line. Avoids parsing and copying these lines.

Closes #13165
2024-04-11 09:29:21 +02:00
Stefan Eissing
c296abd42d
llist: add Curl_llist_append()
- use for better readability in all places where the "insert_next"
  actually performs an append to the list
- add some tests in unit1300

Closes #13336
2024-04-11 09:00:51 +02:00
Stefan Eissing
8cee4c9238
gnutls: lazy init the trust settings
- delay loading of trust anchors and CRLs after the ClientHello
  has been sent off
- add tracing to IO operations
- on IO errors, return the CURLcode of the underlying filter

Closes #13339
2024-04-11 08:59:25 +02:00
Marcel Raad
61e6db87ac
http_negotiate: fix CURL_DISABLE_PROXY build
`proxyuserpwd` was removed from `dynamically_allocated_data` in commit
f46385d36d.

Closes https://github.com/curl/curl/pull/13334
2024-04-10 14:56:05 +02:00
Viktor Szakats
4f15443d2c
quic: fixup duplicate static function name (for cmake unity)
Visible in daily curl-for-win builds:
https://github.com/curl/curl-for-win/actions/runs/8621925870

```
lib/vquic/curl_ngtcp2.c:1916:12: error: redefinition of 'ossl_new_session_cb'
static int ossl_new_session_cb(SSL *ssl, SSL_SESSION *ssl_sessionid)
           ^
lib/vtls/openssl.c:2978:12: note: previous definition is here
static int ossl_new_session_cb(SSL *ssl, SSL_SESSION *ssl_sessionid)
           ^
```
https://github.com/curl/curl-for-win/actions/runs/8621925870/job/23631885439#step:3:6965

Follow-up to 3210101088 #13172
Closes #13332
2024-04-10 10:24:21 +00:00
Viktor Szakats
b445818586
appveyor: make VS2010 job build-only, enable Schannel, fix compiler warnings
Tests were consistently flaky for a while.

Also fix compiler warnings in `CertOpenStore()` calls for old MSVC compilers:
```
C:/projects/curl/lib/vtls/schannel.c(688):
  warning C4306: 'type cast' : conversion from 'int' to 'LPCSTR' of greater size
C:/projects/curl/lib/vtls/schannel_verify.c(642):
  warning C4306: 'type cast' : conversion from 'int' to 'LPCSTR' of greater size
```
Ref: https://ci.appveyor.com/project/curlorg/curl/builds/49580310/job/ywu2y44kymgc0nif#L106

Closes #13330
2024-04-10 07:32:38 +00:00
Daniel Stenberg
6389ba87b8
projects: drop MSVC project files for recent versions
We encourage users to generate visual studio project files using CMake.

We keep project files in git for ancient visual studio versions that
cmake cannot generate files for, but we no longer ship the project files
in the tarballs.

appveyor: switch VisualStudioSolution job to VC12 (Visual Studio 2013)

Co-Authored-by: Viktor Szakats
Co-Authored-by: Jay Satiro

Closes #13311
2024-04-10 07:55:24 +02:00
Viktor Szakats
3d569aaaf8
cmake: use namespaced custom target names
Rename custom target to namespaced (unique) names to avoid colliding
with 3rd-party projects (e.g. libzip) built together with curl.

Reported-by: hammlee96 on github
Fixes #13324
Closes #13326
2024-04-09 21:20:54 +00:00
Viktor Szakats
feb1a3527e
appveyor: re-enable OpenSSL 3, bump to 3.2.1
Ref: b62454a875 #13266
Closes #13329
2024-04-09 21:16:53 +00:00
Stefan Eissing
a6ef4056e8
CI: upgrade openssl version to 3.3.0 for openssl-quic
Closes #13328
2024-04-09 17:41:46 +02:00