The test-ci target now uses 2 processes by default, but the amount of
parallelism is tuned for each CI service and build environment based on
results of a number of test runs. Some CI services use super-
oversubscribed build machines that can barely run the curl tests
already with no parallelism without frequently failing with
timing-induced failures. These continue to be run without parallelism.
Other services provide two fast, unloaded cores and these run with 14
processes, which is a good default for this kind of environment.
Here's a summary of the number of test processes by CI service:
Appveyor - 2 (Windows MSVC), 1 (others)
Azure - 2
Circle CI - 14
Cirrus - 28 (macOS), 14 (Linux), 7 (FreeBSD), 5 (macOS torture), 2 (Windows)
GitHub Actions - 3 (macOS), 2 (Linux)
Some of these are a bit conservative to keep timing-induced flakiness down.
The net result is that the first test results should arrive only
3 minutes after a commit submission.
Changes merged via separate commits:
- 2a7c8b27fd#14171
- 72341068a2
- efce544418#14244
- c6cf411bac
Ref: #10818Closes#11510
This seems to be the only way to see what actual toolchain commands were
run, and with what arguments.
Without `dos2unix`, `cat` output comes out empty.
Closes#13957
Simplify controlling whether to build and/run tests in a CI job.
Apply the TFLAGS='skipall' (do not build nor run tests) or
'skiprun' (build, but do not run) method already used with old-mingw-w64
and msvc jobs to existing Windows jobs in GHA and AppVeyor.
Also:
- add Cygwin/cmake test build and run steps while here.
- replace `DISABLED_TESTS` with `TFLAGS` in AppVeyor.
Closes#13796
Before this patch, `ENABLE_CURLDEBUG` (memory tracking) was
unconditionally enabled when `ENABLE_DEBUGBUILD` was set. This made
testing some build configurations complicated. To fix it, this patch
makes `ENABLE_CURLDEBUG` to receive the value of `ENABLE_DEBUG` by
default, while allowing free override by the user.
This allows to use the config:
`ENABLE_DEBUGBUILD=ON ENABLE_CURLDEBUG=OFF`
to enable debug features, without also enabling memory tracking.
This is important because some other build methods allow to set one of
these features but not the other. This patch allows to test any
combination with CMake.
This makes it unnecessary to use the workaround of passing
`-DDEBUGBUILD` via `CMAKE_C_FLAGS`. Which has the disadvantage that our
CMake logic cannot easily detect it, e.g. for disabling symbol hiding on
Windows for `ENABLE_DEBUG`/`DEBUGBUILD` builds.
Cherry-picked from #13718Closes#13792
Before this patch `ENABLE_DEBUG=ON` always enabled the TrackMemory
(aka `ENABLE_CURLDEBUG=ON`) feature, but required the `Debug` CMake
configration to actually enable curl debug features
(aka `-DDEBUGBUILD`).
Curl debug features do not require compiling with C debug options. This
also made enabling debug features unintuitive and complicated to use.
Due to other issues (subject to PR #13694) it also caused an error in
default (and `Release`/`MinSizeRel`/`RelWithDebInfo`) configs, when
building the `testdeps` target:
```
ld: CMakeFiles/unit1395.dir/unit1395.c.o: in function `test':
unit1395.c:(.text+0x1a0): undefined reference to `dedotdotify'
```
Ref: https://github.com/curl/curl/actions/runs/9037287098/job/24835990826#step:3:2483
Fix it by always defining `DEBUGBUILD` when setting `ENABLE_DEBUG=ON`.
Decoupling this option from the selected CMake configuration.
Note that after this patch `ENABLE_DEBUG=ON` unconditionally enables
curl debug features. These features are insecure and unsuited for
production. Make sure to omit this option when building for production
in default, `Release` (and other not-`Debug`) modes.
Also delete a workaround no longer necessary in GHA CI jobs.
Ref: 1a62b6e68c (2015-03-03)
Ref: #13583Closes#13592
Re-implement old mingw-w64 jobs in GHA. This allows to use the latest
Windows runners, replacing Windows Server 2012 R2 (gcc 6) and Windows
Server 2016 (gcc 7, 9) with Windows Server 2022.
GHA runners are also significantly faster, and allow running tests in
parallel (`-j14`). It also offloads 3 more long-running jobs from
AppVeyor CI.
These jobs download (then cache) the mingw-w64 packages from their
original location, which allows flexibility in choosing which versions
and flavours (win32/POSIX, SEH/DWARF, 64/32-bit) we want to test in CI.
The new jobs use these distros:
- https://sourceforge.net/projects/mingw-w64/files/Toolchains%20targetting%20Win64/Personal%20Builds/mingw-builds/ (for gcc 7, same as on AppVeyor)
- https://sourceforge.net/projects/mingw-w64/files/Toolchains%20targetting%20Win32/Personal%20Builds/mingw-builds/ (for gcc 6, same as on AppVeyor)
- https://winlibs.com/ (for gcc 9)
I matched existing AppVeyor job configs, with these differences:
- gcc 6.4.0 instead of 6.3.0.
(same distro as on AppVeyor, but the latest bugfix release)
- gcc 9.5.0 instead of 9.1.0 and a different (but compatible) binary distro.
(in AppVeyor this relies on an old MSYS2 pre-installed on the runner)
- using win32 builds instead of posix for gcc 6.4.0 and 7.3.0.
- websockets enabled.
- always build examples.
- always build tests (this wasn't done for 6.4.0 with AppVeyor CI).
I did not replicate existing test exclusions, and oddly enough the few
failures (so far) were different from MSYS2 jobs and also from their
AppVeyor CI counterparts.
Also:
- delete redundant (default) `-u` option from `cygpath` calls.
- allow matrix options to override default ones in CMake.
- detect and use Windows-supplied curl for `TFLAGS` `-ac` option.
(it's available in modern runners.)
- delete the 3 AppVeyor CI jobs now replicated in GHA.
- appveyor: prefer `SYSTEMROOT` over `WINDIR`.
- tidy-up quotes.
Job performance:
```
AppVeyor GHA
w/examples
w/tests
-------- ----------
CMake, mingw-w64, gcc 6, Debug, x86, Schannel, Static, no-unity 1m25s 8m50s
CMake, mingw-w64, gcc 7, Debug, x64, Schannel, Static, Unicode 31m45s 9m39s
CMake, mingw-w64, gcc 9, Debug, x64, Schannel, Static 28m25s 13m38s
```
Based on these runs:
https://ci.appveyor.com/project/curlorg/curl/builds/49880799https://github.com/curl/curl/actions/runs/9218292508
Notice that building examples and tests is time consuming.
We can tweak any build parameter as necessary to make them more useful
and/or without clogging the job queue or introducing flakiness.
Closes#13759
- re-implement autotools MSYS and Cygwin AppVeyor jobs in GHA.
Now build with SSL and PSL to improve test coverage.
- re-implement MSYS2 mingw-w64 gcc 13 AppVeyor job in GHA.
`CMake, mingw-w64, gcc 13, Debug, x64, Schannel, Static, Unicode`
- add new cmake Cygwin job (build-only).
- enable `-j14` parallelism when running tests.
- delete the 5 migrated jobs from AppVeyor CI.
- add 2 build-only mingw-w64 builds, gcc Release and clang OpenSSL.
- also enable brotli, libssh2, nghttp2 for more test coverage.
These jobs offer better performance, more flexibility and
parallelization compared to the AppVeyor ones they replace. It also
offloads AppVeyor, allowing to iterate faster. They also appear more
reliable than e.g. Azure Windows jobs, where runners are prone to fail
[1].
Closes#13599
[1]:
`Exit code 143 returned from process: file name 'C:\Windows\system32\docker.EXE',
arguments 'exec -i 6b13a669c6dfe7fb9f59414369872fd64d61c7182f880c3d39c135cb4c115c8f
C:\__a\externals\node\bin\node.exe C:\__w\_temp\containerHandlerInvoker.js'.`
The combination of `-DDEBUGBUILD`, a shared `curl.exe`, and the VS2008
compiler creates a `curl.exe` segfaulting on startup:
```
+ _bld/src/curl.exe --version
./appveyor.sh: line 122: 793 Segmentation fault "${curl}" --version
Command exited with code 139
```
Ref: https://ci.appveyor.com/project/curlorg/curl/builds/49817266/job/651iy6qn1e238pqj#L191
Add job that triggers the issue and add the necessary logic to skip
running the affected `curl.exe`.
Ref: #13592Closes#13654
- delete a duplicate line.
- simplify a `make` call.
- merge two `if` branches.
- reorder autotools options for clarity.
- add `--enable-warnings` where missing (it's also the default.)
- add empty lines to YAML for readability.
- use lowercase install prefix/directory.
Closes#13598
OpenSSL moved directories, and bumped versions in AppVeyor CI.
Downgrading is not an ideal solution, but however trivial the solution
may be, I failed to come with anything that made CMake recognize either
OpenSSL 3.1 or 3.2.
Possibly caused by:
702e8cdca0https://github.com/appveyor/build-images/pull/149Closes#13266
PowerShell works (after a steep development curve), but one property of
it stuck and kept causing unresolvable usability issues: With
`$ErrorActionPreference=Stop`, it does abort on failures, but shows only
the first line of the error message. In `Continue` mode, it shows the
full error message, but doesn't stop on all errors. Another issue is
PowerShell considering any stderr output as if the command failed (this
has been improved in 7.2 (2021-Nov), but fixed versions aren't running
in CI and will not be for a long time in all test images.)
Thus, we're going with bash.
Also:
- use `-j2` with autotools tests, making them finish 5-15 minutes per
job faster.
- omit `POSIX_PATH_PREFIX`.
- use `WINDIR`.
- prefer forward slashes.
Follow-up to: 75078a415d#11999
Ref: #12444Fixes#12560Closes#12572
