Uses scripts/cmp-config.pl two compare two curl_config.h files,
presumbly generated with configure and cmake. It displays the
differences and filters out a lot of known lines we ignore.
The script also shows the matches that were *not* used. Possibly
subjects for removal.
Closes#11964
Left there by accident after adding proper detection for this.
Follow-up to 772f0d8edf#12006
Ref: #11964 (effort to sync cmake detections with autotools)
Closes#12015
Found the root cause of the startup crash in unity builds with Unicode
and TrackMemory enabled at the same time.
We must make sure that the `memdebug.h` header doesn't apply to
`lib/curl_multibyte.c` (as even noted in a comment there.) In unity
builds all headers apply to all sources, including `curl_multibyte.c`.
This probably resulted in an infinite loop on startup.
Exclude this source from unity compilation with TrackMemory enabled,
in both libcurl and curl tool. Enable unity mode for a debug Unicode
CI job to keep it tested. Also delete the earlier workaround that
fully disabled unity for affected builds.
Follow-up to d82b080f63#12005
Follow-up to 3f8fc25720#11095Closes#11928
"TrackMemory" is `ENABLE_DEBUG=ON` (aka `ENABLE_CURLDEBUG=ON`,
aka `-DCURLDEBUG`).
There is an issue with memory tracking and Unicode when built in "unity"
mode, which results in the curl tool crashing right on startup, even
without any command-line option. Interestingly this doesn't happen under
WINE (at least on the system I tested this on), but consistenly happens
on real Windows machines. Crash is 0xC0000374 heap corruption. Both
shared and static curl executables are affected.
This limitation probably won't hit too many people, but it remains
a TODO to find and fix the root cause and drop this workaround.
Example builds and runs:
https://ci.appveyor.com/project/curlorg/curl/builds/48169111/job/17cptxhtpubd7iwj#L313 (static)
https://ci.appveyor.com/project/curlorg/curl/builds/48169111/job/76e1ge758tbyqu9c#L317 (shared)
Follow-up to 3f8fc25720#11095
Ref: #11928Closes#12005
1. Rewrite in PowerShell:
- rewrite MS-DOS batch build script in PowerShell.
- move some bash operations into native PowerShell.
- fixups for PowerShell insisting on failure when a command outputs
something to stderr.
- fix to actually run `curl -V` after every build.
(and exclude ARM64 builds.)
- also say why we skipped `curl -V` if we had to skip.
- fix CMake warnings about unused configuration variables, by adapting
these dynamically for build cases.
- dedupe OpenSSL path into a variable.
- disable `test1451` failing with a warning anyway due to missing python
impacket. (after trying and failing to install impacket)
PowerShell promotes these warnings to errors by PowerShell. We can also
suppress they wholesale if they start causing issues in the future,
like we already to with `autoreconf` and `./configure`.
PowerShell is better than MS-DOS batches, so the hope is this makes it
easier to extend and maintain the AppVeyor build logic. POSIX/bash isn't
supported inline by AppVeyor on Windows build machines, but we are okay
to keep it in an external script, so it's also an option.
2. CI improvements:
- enable tests for a "unity" build job.
- speed-up CI initialization by using shallow clones of the curl repo.
- speed-up CMake MSVC jobs with `TrackFileAccess=false`.
- enable parallelism in `VisualStudioSolution` builds.
- display CMake version before builds.
- always show the CPU in job names.
- tell which jobs are build-only in job names.
- move `TESTING:` value next to `DISABLED_TESTS:` in two jobs.
- add `config.log` (autotools) to dumped logs (need to enable manually).
3. Style:
- use single-quotes in YAML like we do in other CI YAML files.
It also allows to drop quoting characters and lighter to write/read.
(keep double quotes for PowerShell strings needing expansion.)
Closes#11999
- set `HAVE_LDAP_URL_PARSE` if `ldap_url_parse` function exists.
Before this patch we set it based it on the presence of `stricmp`,
which correctly enabled it on e.g. Windows, but was inaccurate for
other platforms.
- always set `HAVE_LDAP_SSL` if an LDAP backend is detected and
LDAPS is not explicitly disabled. This mimics autotools behaviour.
Previously we set it only for Windows LDAP. After this fix, LDAPS is
correctly enabled in default macOS builds.
- enable LDAP[S] for a CMake macOS CI job. Target OS X 10.9 (Mavericks)
to avoid deprecation warnings for LDAP API.
- always detect `HAVE_LDAP_SSL_H`, even with LDAPS explicitly disabled.
This doesn't make much sense, but let's do it to sync behaviour with
autotools.
- fix benign typo in variable name.
Ref: #11964 (effort to sync cmake detections with autotools)
Closes#12006
This restores `CURL_CHECK_FUNC_IOCTL` detection. I deleted it in
4d73854462 and
c3456652a0 (2022-08), because the
`HAVE_IOCTL` result it generated was unused in the source. But,
I did miss the fact that this had two dependent checks:
`CURL_CHECK_FUNC_IOCTL_FIONBIO`,
`CURL_CHECK_FUNC_IOCTL_SIOCGIFADDR` that we do actually need:
`HAVE_IOCTL_FIONBIO`, `HAVE_IOCTL_SIOCGIFADDR`.
Regression from 4d73854462
Ref: #11964 (effort to sync cmake detections with autotools)
Closes#12008
- refs #11982 where it was noted that paused transfers may
close successfully without delivering the complete data
- made sample poc into tests/http/client/h2-pausing.c and
added test_02_27 to reproduce
Closes#11989Fixes#11982
Reported-by: Harry Sintonen
Before this patch CMake builds accepted any value and it was used at
runtime as-is. This patch make sure that the selected default backend
is also enabled in the build. It also enforces a full lowercase value.
This improves reproducibility and brings CMake in sync with autotools
which already worked like described above.
Follow-up to 26c7feb8b9#11774Closes#11998
autotools was using the same value as CMake, but with an ending
slash. Delete the ending slash to match configurations.
Ref: #11964 (effort to sync cmake detections with autotools)
Closes#11997
The default wolfSSL_CTX_load_verify_locations() function is quite picky
with the certificates it loads and will for example return error if just
one of the certs has expired.
With the *_ex() function and its WOLFSSL_LOAD_FLAG_IGNORE_ERR flag, it
behaves more similar to what OpenSSL does by default.
Even the set of default certs on my Debian unstable has several expired
ones.
Assisted-by: Juliusz Sosinowicz
Assisted-by: Michael Osipov
Closes#11987
Based on existing autotools logic.
autotools checks for old versions of the allowlisted target OSes and
disables this feature when seeing them. In CMake we assume we're running
on newer systems and enable regardless of OS version.
autotools always runs all 3 probes for non-fast-tracked systems and
enables this feature if any one of them was successful. To save
configuration time, CMake stops at the first successful check.
OpenBSD is not fast-tracked and then gets blocklisted as a generic BSD
system. I haven't double-checked if this is correct, but looks odd.
Ref: #11964 (effort to sync cmake detections with autotools)
Closes#11979
- run `curl -V` after builds to see if they run and with what features.
Except for one job where a CRT DLL is missing. And ARM64 which should
fail, but is silently not launched instead.
- copy libcurl DLL next to curl tool and tests binaries in shared mode.
This makes it possible to run the tests. (We don't run tests after
these builds yet.)
- list the DLLs and EXEs present after the builds.
- add `DEBUG` variable for CMake builds to allow disabling it, for
testing non-debug builds. (currently enabled for all)
- add commented lines that dump CMake configuration logs for debugging
build/auto-detection issues.
- add gcc version to jobs where missing.
- switch a job to the native MSYS2 mingw-w64 toolchain. This adds gcc 9
to the build mix.
- make `SHARED=OFF` and `OPENSSL=OFF` defaults global.
- delete a duplicate backslash.
Closes#11976
It sometimes happens that a test hangs during a test run and never
returns. The test harness will wait indefinitely for the results and on
CI servers the CI job will eventually be killed after an hour or two.
At the end of a test run, if results haven't come in within a couple of
minutes, display the status of all test runners and what tests they're
running to help in debugging the problem.
This feature is really only kick in with parallel testing enabled, which
is fine because without parallel testing it's usually easy to tell what
test has hung.
Closes#11980
in FAQ and MANUAL.md
- example.com was made for this purpose.
- reduces the risk that one of those domains suddenly start hosting
something nasty and we provide links to them
Closes#11986
The default system truststore on FreeBSD has been /etc/ssl/certs for many
years now. It is managed canonically through certctl(8) and contains hashed
symlinks for OpenSSL and other TLS providers.
The previous ones require security/ca_root_nss which might not be installed or
will not contain any custom CA certificates.
Closes#11985
`basename` is present in mingw-w64, missing from MSVC. Pre-cache
accordingly to make configure faster.
Notice that `basename` has a bug so we later disable it even with
mingw-w64:
781242ffa4/lib/curl_setup.h (L820-L825)Closes#11974
- check for arc4random. To make rand.c use it accordingly.
- check for fcntl
- fix fseek detection
- add SIZEOF_CURL_SOCKET_T
- fix USE_UNIX_SOCKETS
- define HAVE_SNPRINTF to 1
- check for fnmatch
- check for sched_yield
- remove HAVE_GETPPID duplicate from curl_config.h
- add HAVE_SENDMSG
Ref: #11964
Co-authored-by: Viktor Szakats
Closes#11973
1. References to curl symbols are now checked that they indeed exist as
man pages. This for \f references as well as the names referenced in the
SEE ALSO section.
Allowlist curl.1 since it is not always built in builds
2. References to curl symbols that lack section now causes warning, since that
will prevent them from getting linked properly
3. Check for "bare" references to curl functions and warn, they should be
references
Closes#11963
With new option `CURL_DISABLE_SRP=ON` to force-disable it.
To match existing option and detection logic in autotools.
Also:
- fix detecting GnuTLS.
We assume `nettle` as a GnuTLS dependency.
- add CMake GnuTLS CI job.
- bump AppVeyor CMake OpenSSL MSVC job to OpenSSL 1.1.1 (from 1.0.2)
TLS-SRP fails to detect with 1.0.2 due to an OpenSSL header bug.
- fix compiler warning when building with GnuTLS and disabled TLS-SRP.
- fix comment typos, whitespace.
Ref: #11964Closes#11967
Earlier this year we changed our own stderr variable to use the standard
name `stderr` (to avoid bugs where someone is using `stderr` instead of
the curl-tool specific variable). This solution needed to override the
standard `stderr` symbol via the preprocessor. This in turn didn't play
well with unity builds and caused curl tool to crash or stay silent due
to an uninitialized stderr. This was a hard to find issue, fixed by
manually breaking out one file from the unity sources.
To avoid two these two tricks, this patch implements a different
solution: Restore using our own local variable for our stderr output and
leave `stderr` as-is. To avoid using `stderr` by mistake, add a
`checksrc` rule (based on logic we already used in lib for `strerror`)
that detects any `stderr` use in `src` and points to using our own
variable instead: `tool_stderr`.
Follow-up to 06133d3e9b
Follow-up to 2f17a9b654Closes#11958
- move definitions from content_encoding.h to sendf.h
- move create/cleanup/add code into sendf.c
- installed content_encoding writers will always be called
on Curl_client_write(CLIENTWRITE_BODY)
- Curl_client_cleanup() frees writers and tempbuffers from
paused transfers, irregardless of protocol
Closes#11908
Curl_timediff rounds down to the millisecond, so curl_multi_perform can
be called too early, then we get a timeout of 0 and call it again.
The code already handled the case of timeouts which expired less than
1ms in the future. By rounding up, we make sure we will never ask the
platform to wake up too early.
Closes#11938
