If a malicious server can trigger a NULL dereference in curl or
otherwise cause curl to crash (and nothing worse), chances are big that
we do not consider that a security problem.
Closes#13974
Multipath TCP (MPTCP), standardized in RFC8684 [1], is a TCP extension
that enables a TCP connection to use different paths.
Multipath TCP has been used for several use cases. On smartphones, MPTCP
enables seamless handovers between cellular and Wi-Fi networks while
preserving established connections. This use-case is what pushed Apple
to use MPTCP since 2013 in multiple applications [2]. On dual-stack
hosts, Multipath TCP enables the TCP connection to automatically use the
best performing path, either IPv4 or IPv6. If one path fails, MPTCP
automatically uses the other path.
To benefit from MPTCP, both the client and the server have to support
it. Multipath TCP is a backward-compatible TCP extension that is enabled
by default on recent Linux distributions (Debian, Ubuntu, Redhat, ...).
Multipath TCP is included in the Linux kernel since version 5.6 [3]. To
use it on Linux, an application must explicitly enable it when creating
the socket. No need to change anything else in the application.
This attached patch adds an --mptcp option which allows the creation of
an MPTCP socket instead of TCP on Linux. If Multipath TCP is not
supported on the system, an error will be reported. It is important to
note that if the end server doesn't support MPTCP, the connection will
continue after a seamless fallback to TCP.
Link: https://www.rfc-editor.org/rfc/rfc8684.html [1]
Link: https://www.tessares.net/apples-mptcp-story-so-far/ [2]
Link: https://www.mptcp.dev [3]
Co-developed-by: Dorian Craps (@CrapsDorian) <doriancraps@gmail.com>
Co-developed-by: Olivier Bonaventure (@obonaventure) <Olivier.Bonaventure@uclouvain.be>
Co-developed-by: Matthieu Baerts (@matttbe) <matttbe@kernel.org>
Signed-off-by: Dorian Craps <dorian.craps@student.vinci.be>
Closes#13278
Add new job to test building for UWP (aka `CURL_WINDOWS_APP`).
Fix fallouts when building for UWP:
- rand: do not use `BCryptGenRandom()`.
- cmake: disable using win32 LDAP.
- cmake: disable telnet.
- version_win32: fix code before declaration.
- schannel: disable `HAS_MANUAL_VERIFY_API`.
- schannel: disable `SSLSUPP_PINNEDPUBKEY`
and make `schannel_checksum()` a stub.
Ref: e178fbd40a#1429
- schannel: make `cert_get_name_string()` a failing stub.
- system_win32: make `Curl_win32_impersonating()` a failing stub.
- system_win32: try to fix `Curl_win32_init()` (untested).
- threads: fix to use `CreateThread()`.
- src: disable searching `PATH` for the CA bundle.
- src: disable bold text support and capability detection.
- src: disable `getfiletime()`/`setfiletime()`.
- tests: make `win32_load_system_library()` a failing stub.
- tests/server/util: make it compile.
- tests/server/sockfilt: make it compile.
- tests/lib3026: fix to use `CreateThread()`.
See individual commits for build error details.
Some of these fixes may have better solutions, and some may not work
as expected. The goal of this patch is to make curl build for UWP.
Closes#13870
Introduce new notation for CURLOPT_INTERFACE / --interface:
ifhost!<interface>!<host>
Binding to an interface doesn't set the address, and an interface can
have multiple addresses.
When binding to an address (without interface), the kernel is free to
choose the route, and it can route through any device that can access
the target address, not necessarily the one with the chosen address.
Moreover, it is possible for different interfaces to have the same IP
address, on which case we need to provide a way to be more specific.
Factor out the parsing part of interface option, and add unit tests:
1663.
Closes#13719
CMake configure doesn't fail often, but when it does, it helps to see
its `CMakeFiles/CMakeConfigureLog.yaml` output. This file is present
since CMake v3.26:
https://cmake.org/cmake/help/v3.26/manual/cmake-configure-log.7.html
(Older CMake versions save similar contend to
`CMakeFiles\CMakeOutput.log` and
`CMakeFiles\CMakeError.log`. This patch doesn't deal with that because
the workflows touched are all running a newer CMake.)
After this patch, we dump the content if cmake fails. Syncing this with
autotools, where we already did that.
Closes#13872
- add special sauce to disable unwanted peer verification by mbedtls
when negotiating TLS v1.3
- add special sauce for MBEDTLS_ERR_SSL_RECEIVED_NEW_SESSION_TICKET
return code on *writing* TLS data. We assume the data had not been
written and EAGAIN.
- return correct Curl error code when peer verification failed.
- disable test_08_05 with 50 HTTP/1.1 connections, as mbedtls reports a
memory allocation failed during handshake.
- bump CI mbedtls version to 3.6.0
Fixes#13653Closes#13838
This toolchain resides in the `mingw32` directory. Make sure to
configure `PATH` accordingly.
Before this patch, it pointed to a non-existing `mingw64` directory,
making the job use the wrong compiler (gcc 12, 64-bit).
Follow-up to e838b341a0#12927Closes#13863
- bump rest of the workflows (windows, macos, distrocheck).
- non-native virtualized envs have 2 CPUs, bump down accordingly.
(for `vmactions/omnios-vm` it's just a guess.)
- bump all to nproc + 1.
Follow-up to e838b341a0#12927Closes#13807
- stop altering the `PATH` via `GITHUB_ENV`. This confused the
`actions/cache` post-job, which needs to run in the exact same
environment as its pre-job, to have a consistent cache entry "version"
hash. Altering the `PATH` via `GITHUB_ENV` spills into the the
post-job and breaks this hash. GHA doesn't reset the env automatically
and I have not found a way to do it manually.
- add double-quotes where missing.
- move cache directory under `USERPROFILE` to not rely on absolute
paths.
- make cache directory flatter and versionless.
Follow-up to 0914d8aadd#13759Closes#13856
This commit unifies the following http3 workflows into http3-linux.yml:
- ngtcp2-linux.yml
- osslq-linux.yml
- quiche-linux.yml
The idea is better use of the build cache. Previously, they
independently create caches with the same key. Some of the caches
include source code and intermediate object files, which makes cache
quite large. In this commit, only built artifacts are cached, which
drastically reduces the cache size. OpenSSL v3, mod_h2 and quiche caches
still include all stuff, but they are left for the later improvement.
Because the contents of the cache have been changed, the cache keys are
also changed to include the word "http3".
Closes#13841
These exclusions came from the AppVeyor CI config, but they do pass now
and they are static tests with no flakiness risk.
Follow-up to 0914d8aadd#13759Closes#13817
`CURLDEBUG` is meant to enable memory tracking, but in a bunch of cases,
it was protecting debug features that were supposed to be guarded with
`DEBUGBUILD`.
Replace these uses with `DEBUGBUILD`.
This leaves `CURLDEBUG` uses solely for its intended purpose: to enable
the memory tracking debug feature.
Also:
- autotools: rely on `DEBUGBUILD` to enable `checksrc`.
Instead of `CURLDEBUG`, which worked in most cases because debug
builds enable `CURLDEBUG` by default, but it's not accurate.
- include `lib/easyif.h` instead of keeping a copy of a declaration.
- add CI test jobs for the build issues discovered.
Ref: https://github.com/curl/curl/pull/13694#issuecomment-2120311894Closes#13718
It affected cmake-unity shared-curltool curldebug mingw-w64 gcc builds
when building the `testdeps` target.
Apply the solution already used in `lib/base64.c` and `lib/dynbuf.c`
to fix it.
Also update an existing GHA CI job to test the issue fixed.
```
In file included from curl/lib/version_win32.c:35,
from curl/_bld/src/CMakeFiles/curl.dir/Unity/unity_0_c.c:145:
curl/lib/memdebug.h:52:14: error: redundant redeclaration of 'curl_dbg_logfile' [-Werror=redundant-decls]
52 | extern FILE *curl_dbg_logfile;
| ^~~~~~~~~~~~~~~~
In file included from curl/src/slist_wc.c:32,
from curl/_bld/src/CMakeFiles/curl.dir/Unity/unity_0_c.c:4:
curl/lib/memdebug.h:52:14: note: previous declaration of 'curl_dbg_logfile' with type 'FILE *' {aka 'struct _iobuf *'}
52 | extern FILE *curl_dbg_logfile;
| ^~~~~~~~~~~~~~~~
curl/lib/memdebug.h:55:44: error: redundant redeclaration of 'curl_dbg_malloc' [-Werror=redundant-decls]
55 | CURL_EXTERN ALLOC_FUNC ALLOC_SIZE(1) void *curl_dbg_malloc(size_t size,
| ^~~~~~~~~~~~~~~
curl/lib/memdebug.h:55:44: note: previous declaration of 'curl_dbg_malloc' with type 'void *(size_t, int, const char *)' {aka 'void *(long long unsigned int, int, const char *)'}
55 | CURL_EXTERN ALLOC_FUNC ALLOC_SIZE(1) void *curl_dbg_malloc(size_t size,
| ^~~~~~~~~~~~~~~
[...]
curl/lib/memdebug.h:110:17: error: redundant redeclaration of 'curl_dbg_fclose' [-Werror=redundant-decls]
110 | CURL_EXTERN int curl_dbg_fclose(FILE *file, int line, const char *source);
| ^~~~~~~~~~~~~~~
curl/lib/memdebug.h:110:17: note: previous declaration of 'curl_dbg_fclose' with type 'int(FILE *, int, const char *)' {aka 'int(struct _iobuf *, int, const char *)'}
110 | CURL_EXTERN int curl_dbg_fclose(FILE *file, int line, const char *source);
| ^~~~~~~~~~~~~~~
```
Ref: https://ci.appveyor.com/project/curlorg/curl/builds/49840554/job/a4aoet17e9qnqx1a#L362
After: https://ci.appveyor.com/project/curlorg/curl/builds/49843735/job/hbo2uah2vj0ns523
Ref: #13689 (CI testing this PR with `DEBUGBUILD`/`CURLDEBUG`/shared-static combinations)
Depends-on: #13694
Depends-on: #13800Closes#13705
For better build performance. Dependency tracking causes a build
overhead while compiling to help a subsequent build, but in CI there is
never one and the extra work is discarded.
Closes#13794
Simplify controlling whether to build and/run tests in a CI job.
Apply the TFLAGS='skipall' (do not build nor run tests) or
'skiprun' (build, but do not run) method already used with old-mingw-w64
and msvc jobs to existing Windows jobs in GHA and AppVeyor.
Also:
- add Cygwin/cmake test build and run steps while here.
- replace `DISABLED_TESTS` with `TFLAGS` in AppVeyor.
Closes#13796
Before this patch `ENABLE_DEBUG=ON` always enabled the TrackMemory
(aka `ENABLE_CURLDEBUG=ON`) feature, but required the `Debug` CMake
configration to actually enable curl debug features
(aka `-DDEBUGBUILD`).
Curl debug features do not require compiling with C debug options. This
also made enabling debug features unintuitive and complicated to use.
Due to other issues (subject to PR #13694) it also caused an error in
default (and `Release`/`MinSizeRel`/`RelWithDebInfo`) configs, when
building the `testdeps` target:
```
ld: CMakeFiles/unit1395.dir/unit1395.c.o: in function `test':
unit1395.c:(.text+0x1a0): undefined reference to `dedotdotify'
```
Ref: https://github.com/curl/curl/actions/runs/9037287098/job/24835990826#step:3:2483
Fix it by always defining `DEBUGBUILD` when setting `ENABLE_DEBUG=ON`.
Decoupling this option from the selected CMake configuration.
Note that after this patch `ENABLE_DEBUG=ON` unconditionally enables
curl debug features. These features are insecure and unsuited for
production. Make sure to omit this option when building for production
in default, `Release` (and other not-`Debug`) modes.
Also delete a workaround no longer necessary in GHA CI jobs.
Ref: 1a62b6e68c (2015-03-03)
Ref: #13583Closes#13592
Continuing the theme, add 3 MSVC jobs with tests, matching
configurations used on AppVeyor. MSVC versions are identical:
19.39.33523.0 + Windows SDK 10.0.22621.0.
Also enable websockets, and build examples. Tests are run in parallel
(`-j14`), with improved performance.
Job performance:
```
AppVeyor GHA
w/examples
-------- ----------
CMake, VS2022, Debug, x64, Schannel, Static, Unicode 38m 4s 11m57s
CMake, VS2022, Debug, x64, no SSL, Static 35m15s 12m 6s
CMake, VS2022, Debug, x64, no SSL, Static, HTTP only 25m25s 10m36s
```
Based on these runs:
https://ci.appveyor.com/project/curlorg/curl/builds/49884748https://github.com/curl/curl/actions/runs/9229448468
This is the first time examples are built in CI with MSVC: Fix all
warnings and errors that came up via
d4b8589055#13771.
Closes#13766
Re-implement old mingw-w64 jobs in GHA. This allows to use the latest
Windows runners, replacing Windows Server 2012 R2 (gcc 6) and Windows
Server 2016 (gcc 7, 9) with Windows Server 2022.
GHA runners are also significantly faster, and allow running tests in
parallel (`-j14`). It also offloads 3 more long-running jobs from
AppVeyor CI.
These jobs download (then cache) the mingw-w64 packages from their
original location, which allows flexibility in choosing which versions
and flavours (win32/POSIX, SEH/DWARF, 64/32-bit) we want to test in CI.
The new jobs use these distros:
- https://sourceforge.net/projects/mingw-w64/files/Toolchains%20targetting%20Win64/Personal%20Builds/mingw-builds/ (for gcc 7, same as on AppVeyor)
- https://sourceforge.net/projects/mingw-w64/files/Toolchains%20targetting%20Win32/Personal%20Builds/mingw-builds/ (for gcc 6, same as on AppVeyor)
- https://winlibs.com/ (for gcc 9)
I matched existing AppVeyor job configs, with these differences:
- gcc 6.4.0 instead of 6.3.0.
(same distro as on AppVeyor, but the latest bugfix release)
- gcc 9.5.0 instead of 9.1.0 and a different (but compatible) binary distro.
(in AppVeyor this relies on an old MSYS2 pre-installed on the runner)
- using win32 builds instead of posix for gcc 6.4.0 and 7.3.0.
- websockets enabled.
- always build examples.
- always build tests (this wasn't done for 6.4.0 with AppVeyor CI).
I did not replicate existing test exclusions, and oddly enough the few
failures (so far) were different from MSYS2 jobs and also from their
AppVeyor CI counterparts.
Also:
- delete redundant (default) `-u` option from `cygpath` calls.
- allow matrix options to override default ones in CMake.
- detect and use Windows-supplied curl for `TFLAGS` `-ac` option.
(it's available in modern runners.)
- delete the 3 AppVeyor CI jobs now replicated in GHA.
- appveyor: prefer `SYSTEMROOT` over `WINDIR`.
- tidy-up quotes.
Job performance:
```
AppVeyor GHA
w/examples
w/tests
-------- ----------
CMake, mingw-w64, gcc 6, Debug, x86, Schannel, Static, no-unity 1m25s 8m50s
CMake, mingw-w64, gcc 7, Debug, x64, Schannel, Static, Unicode 31m45s 9m39s
CMake, mingw-w64, gcc 9, Debug, x64, Schannel, Static 28m25s 13m38s
```
Based on these runs:
https://ci.appveyor.com/project/curlorg/curl/builds/49880799https://github.com/curl/curl/actions/runs/9218292508
Notice that building examples and tests is time consuming.
We can tweak any build parameter as necessary to make them more useful
and/or without clogging the job queue or introducing flakiness.
Closes#13759
We do it in Cirrus CI, but for some platforms it's not possible to
delete it and tests work anyway.
The test runner also runs `../src/curl` by default, which is always the
one freshly built. The runner may also need the system curl to talk to
APIs when needed.
Also:
- stop setting `CURL` env. This isn't picked up by the runners,
and works out of the box anyway.
- quote an option just in case.
Follow-up to 90e644f944#13583Closes#13765
