Move the listen/accept handling of the FTP active data connection
into the socket filter and monitor 'connected' status of that as
with passive connections - more or less.
The advantage is that the socket filter now reports being connected
only when the server has actually called and accept() has been done.
This enables to bootstrap the filter chain on the data connection
just like any other. A require SSL filter can then be added right
at the start and does not need to be patched in later.
Still, the active connection keeps on needing special handling in
ftp.c as the control connection needs to be monitored while waiting
as the server might send error responses this way. So, things did
not turn out quite as squeaky clean as hoped for, but still seems
better to do that way.
Closes#14798
For TLSv1.3, if supported, observer special return code to retrieve
newly arrived session from mbedTLS.
Adjust test expectations now that TLSv1.3 session resumption works in
mbedTLS >= 3.6.0.
Based on #14135 by @ad-chaos
Closes#15245
Register a callback to get notified of new SSL sessions by wolfSSL.
Remove the explicit session retrieval after handshake, since this does
not work for TLSv1.3.
Adjust test expectations now that TLSv1.3 session resumption works
in wolfSSL.
Closes#15243
Replace with namespaced local macro `Curl_getpid()`.
Redefining symbols can backfire if that symbol is used in system
headers, especially with unity build. We haven't seen a fallout in CI
or supported envs, but do it anyway for good measure.
Bug report: https://datagirl.xyz/posts/wolfssl_curl_w2k.htmlCloses#15263
- Linux: move test and pytest prereqs right before test run.
- returns build phase results faster.
- allows skipping steps for jobs that don't need them.
- makes dependencies more transparent.
- sync prereq install step names.
- use `tests/requirements.txt` more.
Closes#15275
- limit `SIZEOF_SA_FAMILY_T` detection to non-Windows.
- make sure `sys/socket.h` exists before detecting `SIZEOF_SA_FAMILY_T`.
- limit `mach_absolute_time()` detection to `APPLE`. Drop from Windows
pre-cache.
- skip `HAVE_LIBSOCKET` detection for Windows, drop pre-cached value.
- drop redundant pre-cached `HAVE_LIBZ` for Windows.
- `curl_required_libpaths()`: stop accepting multiple arguments.
To prepare for `CMAKE_REQUIRED_LINK_DIRECTORIES` support.
Follow-up to 7bab201abe#15193
- GSS: fix recent rebase mistakes:
- fix variable name.
- do not add a header twice.
Follow-up to 91d451b488#15157
- GSS: quote a variable.
Closes#15271
- bump github/codeql-action from 3.26.10 to 3.26.11
- bump vmactions/omnios-vm from 1.0.7 to 1.0.8
- bump actions/cache from 4.0.2 to 4.1.1
Closes#15178Closes#15179Closes#15244
Add session reuse for QUIC transfers using GnuTLS. This does not include
support for TLS early data, yet.
Fix check of early data support in common GnuTLS init code to not access
the filter context, as the struct varies between TCP and QUIC
connections.
Closes#15265
This indicates that the file was created by curl which can help a user
determine the origin of a file. Like the other attributes, this is only
enabled with the --xattr option.
Closes#15237
- reduce `check_include_file_concat()` use to those headers that either
depend on a previously detected header, or another header or symbol
detection depend on it.
- replace `check_symbol_exists()` with `check_function_exists()` for
functions that are detected with `AC_CHECK_FUNCS()` in `./configure`.
This makes `setmode()` no longer be detected with MSYS, syncing
this with `./configure`. Instead `_setmode()` is used now also in
CMake MSYS builds. This is consistent with Cygwin builds also.
- add comment about which header/symbol detection depends on what
header. Based on `./configure` mainly.
- form `CURL_TEST_DEFINES` manually, and include only those macros which
are actually used in `CMake/CurlTests.c`.
- change `curl_internal_test()` to use `CMAKE_REQUIRED_DEFINITIONS`,
instead of `CMAKE_REQUIRED_FLAGS` to simplify the logic, and to allow
dropping the latter macro completely.
- drop `windows.h` from header and symbol checks.
- `./configure`: add comment about whether `netinet/in6.h`, `sys/un.h`
are indeed meant to be included for all detections. There is a chance
they were added there by accident.
Detection resuls were cross-checked between
436bbbe7ab (master) and
48ff4694e608ccfdedf7ce5bab2b96d6b2c23cda (this PR), for CI GHA Linux,
Linux HTTP/3, non-native, macOS and Windows jobs.
Closes#15164
Based on #14135, implement TLSv1.3 earlydata support for the curl
command line, libcurl and its implementation in GnuTLS.
If a known TLS session announces early data support, and the feature is
enabled *and* it is not a "connect-only" transfer, delay the TLS
handshake until the first request is being sent.
- Add --tls-earldata as new boolean command line option for curl.
- Add CURLSSLOPT_EARLYDATA to libcurl to enable use of the feature.
- Add CURLINFO_EARLYDATA_SENT_T to libcurl, reporting the amount of
bytes sent and accepted/rejected by the server.
Implementation details:
- store the ALPN protocol selected at the SSL session.
- When reusing the session and enabling earlydata, use exactly
that ALPN protocol for negoptiation with the server. When the
sessions ALPN does not match the connections ALPN, earlydata
will not be enabled.
- Check that the server selected the correct ALPN protocol for
an earlydata connect. If the server does not confirm or reports
something different, the connect fails.
- HTTP/2: delay sending the initial SETTINGS frames during connect,
if not connect-only.
Verification:
- add test_02_32 to verify earlydata GET with nghttpx.
- add test_07_70 to verify earlydata PUT with nghttpx.
- add support in 'hx-download', 'hx-upload' clients for the feature
Assisted-by: ad-chaos on github
Closes#15211
- split the huge Curl_cookie_add() into several smaller static functions
- switch to using the common llist instead of custom linked list
- use less memory for *getlist()
- use bitfields for flags in the Cookie struct
- avoid the copy for date parsing
- more consistent variable naming
Closes#15247
Replace `check_include_file_concat()` with `check_include_file()` in
GSS/LDAP detection to avoid these headers spilling into subsequent
feature checks.
- For LDAP, reverse detection order to match with `./configure`.
Though, in current LDAP packages `ldap.h` does include `lber.h`.
- For GSS, align header detection logic with `./configure`, where
`gssapi/gssapi_generic.h` might require `gssapi/gssapi.h`, and
`gssapi/gssapi_krb5.h` might require both.
Ref: #436Closes#15157
Via these configuration values:
- `LDAP_LIBRARY`
- `LDAP_LBER_LIBRARY`
- `LDAP_INCLUDE_DIR`
Following the naming scheme used in `Find` modules.
Cherry-picked from #15157Closes#15255
Add comments saying when we want values set in feature check option
variables to apply to all feature checks, globally. These are currently:
`ws2_32` and `socket` libraries, and `-D_WIN32_WINNT=` macro.
Also use `list(APPEND ...)` for the libraries to avoid overwriting
potentially existing values.
Cherry-picked from #15157Closes#15253
It was done for `zlib`, `brotli`, `libpsl`, `libssh2`, `wolfssh`
(a copy-paste case for `wolfssh`).
Feature detections should not rely by default on dependency headers.
There is no evidence they do now. If it becomes necessary, headers
should added for the duration of the feature check.
Ref: 118977f19d
Cherry-picked from #15157Closes#15252
Enclose
`CMAKE_EXTRA_INCLUDE_FILES`,
`CMAKE_REQUIRED_DEFINITIONS`,
`CMAKE_REQUIRED_FLAGS`,
`CMAKE_REQUIRED_INCLUDES`,
`CMAKE_REQUIRED_LIBRARIES`,
`CMAKE_REQUIRED_LINK_OPTIONS`,
settings within `cmake_push_check_state()`/`cmake_pop_check_state()`
calls. It prevents spilling them into other feature checks. It also
replaces manual resets found in some places (which can have
the undesired side-effect of destroying values meant for global use.)
Cherry-picked from #15157Closes#15251
Also:
- detect and add required system libraries for Rustls on macOS and
non-Windows.
- add Linux CMake jobs for the touched dependencies.
Caveats:
- MSH3 generates a broken `libmsh3.pc`, so needs manual config.
Upstream PR: https://github.com/nibanks/msh3/pull/225
- Rustls `.pc` file missing, so needs manual config.
An internal change worthy of mention is that we are using the lib path
and name information returned by `pkg-config` as-is. Meaning the libname
doesn't include the full path, like it's usual with native cmake
detection. The path comes separately and needs to be rolled separately.
For this we add it to targets via `link_directories()`. We also keep tab
of them in `CURL_LIBDIRS` and use that in `libcurl.pc`. Feature checks
also need to receive these paths. CMake doesn't offer
a `CMAKE_REQUIRED_*` variable for this purpose, only
a `CMAKE_REQUIRED_LINK_OPTIONS` accepting raw linker flags. Add a macro
to convert a list of paths to linker options to solve it. wolfSSL
requires this for now.
Closes#15193
- use shallow clone for submodules.
- reduce total job timeout from 90/60 -> 45 minutes.
- use `$HOME` instead of literal.
- http3-linux: sync step yaml order with linux.yml.
- http3-linux: add cmake + ninja support like in linux.yml.
- http3-linux: dump confgure log, test config, curl -V like in linux.yml.
- http3-linux: skip restoring gnutls and wolfssl when not used.
- dump `curl_config.h`.
- fold a long line.
Closes#15242
CMake builds mbedTLS in Debug mode by default, which was the reason
for these consistent test failures:
```
FAIL 1631: 'FTP through HTTPS-proxy' FTP, HTTPS-proxy
FAIL 1632: 'FTP through HTTPS-proxy, with connection reuse' FTP, HTTPS-proxy
```
Sometimes also:
```
FAIL 303: 'HTTPS with 8 secs timeout' HTTPS, HTTP GET, timeout, FAILURE
```
https://github.com/curl/curl/actions/runs/11260616621/job/31313234198
Fix it by building in `RelWithDebInfo` mode, matching the bare
`Makefile` builds used earlier. (`Release` mode also works.)
Cache sizes:
- Makefile: 10MB
- CMake Release: 1MB
- CMake RelWithDebInfo: 2.5MB
Ref: #15215
Follow-up to e377c91766#15208Closes#15238
- on load, only replace existing HSTS entries if there is a full host
match
- on matching, prefer a full host match and secondary the longest tail
subdomain match
Closes#15210
When a multi handle is being cleaned up, it can still cause user
callbacks to be fired. Notably Curl_cpool_destroy calls socket_callback
on all pooled connections. It's still possible for the callback to call
curl_multi_assign leading to an assert.
This commit moves clearing of a multi handle magic to a point where the
multi handle stops being a valid object.
Fixes#15201Closes#15206
- update mbedTLS repo URL.
- switch local mbedTLS build to use CMake, and Ninja.
CMake build is required to create and install mbedTLS `pkg-config`
files. (as of v3.6.1)
`-DCMAKE_POSITION_INDEPENDENT_CODE=ON` required to avoid this error
when linking mbedtls to `libcurl.so`:
```
/usr/bin/ld: /home/runner/mbedtls/lib/libmbedcrypto.a(cipher.c.o): warning: relocation against `mbedtls_cipher_base_lookup_table' in read-only section `.text'
/usr/bin/ld: /home/runner/mbedtls/lib/libmbedtls.a(ssl_tls.c.o): relocation R_X86_64_PC32 against symbol `mbedtls_x509_crt_profile_suiteb' can not be used when making a shared object; recompile with -fPIC
/usr/bin/ld: final link failed: bad value
```
Ref: https://github.com/curl/curl/actions/runs/11245069259/job/31264386723#step:40:43
- make local mbedTLS build 10x smaller by omitting programs and tests.
- GHA/linux: fix cmake warning by adding `-B .` option.
- GHA/linux: add build-only cmake job for packaged mbedTLS (2.x).
- fix compiler warning when building with mbedTLS 2.x:
```
/home/runner/work/curl/curl/lib/vtls/mbedtls.c:344:1: error: ‘mbed_cipher_suite_get_str’ defined but not used [-Werror=unused-function]
344 | mbed_cipher_suite_get_str(uint16_t id, char *buf, size_t buf_size,
| ^~~~~~~~~~~~~~~~~~~~~~~~~
```
Ref: https://github.com/curl/curl/actions/runs/11244999065/job/31264168295#step:40:50
Also in preparation for #15193Closes#15208
