This version adds an important feature that will allow more PRs to be
labelled. Rather than being limited to labeling PRs with files that
match a single glob, it can now label them if multiple changed files
match any one of a number of globs.
- set CURL_CI for pytest runs in CI environments
- exclude timing sensitive tests from CI runs
- for failed results, list only the log and stat of
the failed transfer
- fix type in http.c comment
Closes#11812
ngtcp2 v0.19.0 made size of `ecn` member of `ngtcp2_pkt_info`
an `uint8_t` (was: `uint32_t`). Adjust our local cast accordingly.
Fixes:
```
./curl/lib/vquic/curl_ngtcp2.c:1912:12: warning: implicit conversion loses integer precision: 'uint32_t' (aka 'unsigned int') to 'uint8_t' (aka 'unsigned char') [-Wimplicit-int-conversion]
pi.ecn = (uint32_t)ecn;
~ ^~~~~~~~~~~~~
```
Also bump ngtcp2, nghttp3 and nghttp2 to their latest versions in our
docs and CI.
Ref: 80447281bb
Ref: https://github.com/ngtcp2/ngtcp2/pull/877Closes#11798
This is done by cross-compiling under regular x86_64 Linux. Since the
kernel offers backwards compatibility, the binaries can be tested as
normal.
Closes#11799
- added test cases for various code paths
- fixed handling of blocked write when stream had
been closed inbetween attempts
- re-enabled DEBUGASSERT on send with smaller data size
- in debug builds, environment variables can be set to simulate a slow
network when sending data. cf-socket.c and vquic.c support
* CURL_DBG_SOCK_WBLOCK: percentage of send() calls that should be
answered with a EAGAIN. TCP/UNIX sockets.
This is chosen randomly.
* CURL_DBG_SOCK_WPARTIAL: percentage of data that shall be written
to the network. TCP/UNIX sockets.
Example: 80 means a send with 1000 bytes would only send 800
This is applied to every send.
* CURL_DBG_QUIC_WBLOCK: percentage of send() calls that should be
answered with EAGAIN. QUIC only.
This is chosen randomly.
Closes#11756
- delete completed TODO from `./CMakeLists.txt`.
- convert a C++ comment to C89 in `./CMake/CurlTests.c`.
- delete duplicate EOLs from EOF.
- add missing EOL at EOF.
- delete whitespace at EOL (except from expected test results).
- convert tabs to spaces.
- convert CRLF EOLs to LF in GHA yaml.
- text casing fixes in `./CMakeLists.txt`.
- fix a codespell typo in `packages/OS400/initscript.sh`.
Closes#11772
- Change curl version and os sections from single-line input to
multi-line textarea.
- Require curl version and os sections to be filled out before report
can be submitted.
Closes https://github.com/curl/curl/pull/11636
Add `test1279` to verify that `libcurl.def` lists all exported API
functions found in libcurl headers.
Also:
- extend test suite XML `stdout` tag with the `loadfile` attribute.
- fix `tests/extern-scan.pl` and `test1135` to include websocket API.
- use all headers (sorted) in `test1135` instead of a manual list.
- add options `--sort`, `--heading=` to `tests/extern-scan.pl`.
- add `libcurl.def` to the auto-labeler GHA task.
Follow-up to 2ebc74c36aCloses#11570
This patch adds the ability to build a static and shared libcurl library
in a single build session. It also adds an option to select which one to
use when building the curl executable.
New build options:
- `BUILD_STATIC_LIBS`. Default: `OFF`.
Enabled automatically if `BUILD_SHARED_LIBS` is `OFF`.
- `BUILD_STATIC_CURL`. Default: `OFF`.
Requires `BUILD_STATIC_LIBS` enabled.
Enabled automatically if building static libcurl only.
- `STATIC_LIB_SUFFIX`. Default: empty.
- `IMPORT_LIB_SUFFIX`. Default: `_imp` if implib filename would collide
with static lib name (typically with MSVC) in Windows builds.
Otherwise empty.
Also:
- Stop setting the `CURL_STATICLIB` macro via `curl_config.h`, and pass
it directly to the compiler. This also allows to delete a condition
from `tests/server/CMakeLists.txt`.
- Complete a TODO by following the logic used in autotools (also for
`LIBCURL_NO_SHARED`), and set `-DCURL_STATICLIB` in `Cflags:` of
`libcurl.pc` for _static-only_ curl builds.
- Convert an existing CI test to build both shared and static libcurl.
Closes#11505
- remove quiche from standard `linux` workflow
- add mod_h2 caching to quiche workflow
- rename quiche to quiche-linux
- move version definitions into env section
Closes#11528
- depending on what is tunneled, the proxy may never get invoked for
receiving data explicitly. Not progressing ingress may lead to stalls
due to missed WINDOW_UPDATEs.
CI:
- add a chache for building mod_h2
Closes#11527
Issue forms allow you to define web-like input forms using YAML
syntax. It allows you to guide the reporter to get the required
information.
Signed-off-by: Jan Macku <jamacku@redhat.com>
Closes#11474
If brew install/update links openssl into /usr/local, it will be found
before anything we add with `-isystem path` to CPP/LDLFAGS. Get rid of
that by unlinking the keg.
Fixes#11413Closes#11436
Most CI services provide at least two cores, so enable parallel make
jobs to take advantage of that for builds. Some dependencies aren't safe
to build in parallel so leave those as-is. Also, rename a few
workflows to eliminate duplicate names and provide a better idea what
they're about.
These two functions were added in 7.44.0 when CURLMOPT_PUSHFUNCTION was
introduced but always lived a life in the shadows, embedded in the
CURLMOPT_PUSHFUNCTION man page. Until now.
It makes better sense and gives more visibility to document them in
their own stand-alone man pages.
Closes#11286
Aka "jumbo" or "amalgamation" builds. It means to compile all sources
per target as a single C source. This is experimental.
You can enable it by passing `-DCMAKE_UNITY_BUILD=ON` to cmake.
It requires CMake 3.16 or newer.
It makes builds (much) faster, allows for better optimizations and tends
to promote less ambiguous code.
Also add a new AppVeyor CI job and convert an existing one to use
"unity" mode (one MSVC, one MinGW), and enable it for one macOS CI job.
Fix related issues:
- add missing include guard to `easy_lock.h`.
- rename static variables and functions (and a macro) with names reused
across sources, or shadowed by local variables.
- add an `#undef` after use.
- add a missing `#undef` before use.
- move internal definitions from `ftp.h` to `ftp.c`.
- `curl_memory.h` fixes to make it work when included repeatedly.
- stop building/linking curlx bits twice for a static-mode curl tool.
These caused doubly defined symbols in unity builds.
- silence missing extern declarations compiler warning for ` _CRT_glob`.
- fix extern declarations for `tool_freq` and `tool_isVistaOrGreater`.
- fix colliding static symbols in debug mode: `debugtime()` and
`statename`.
- rename `ssl_backend_data` structure to unique names for each
TLS-backend, along with the `ssl_connect_data` struct member
referencing them. This required adding casts for each access.
- add workaround for missing `[P]UNICODE_STRING` types in certain Windows
builds when compiling `lib/ldap.c`. To support "unity" builds, we had
to enable `SCHANNEL_USE_BLACKLISTS` for Schannel (a Windows
`schannel.h` option) _globally_. This caused an indirect inclusion of
Windows `schannel.h` from `ldap.c` via `winldap.h` to have it enabled
as well. This requires `[P]UNICODE_STRING` types, which is apperantly
not defined automatically (as seen with both MSVS and mingw-w64).
This patch includes `<subauth.h>` to fix it.
Ref: https://github.com/curl/curl/runs/13987772013
Ref: https://dev.azure.com/daniel0244/curl/_build/results?buildId=15827&view=logs&jobId=2c9f582d-e278-56b6-4354-f38a4d851906&j=2c9f582d-e278-56b6-4354-f38a4d851906&t=90509b00-34fa-5a81-35d7-5ed9569d331c
- tweak unity builds to compile `lib/memdebug.c` separately in memory
trace builds to avoid PP confusion.
- force-disable unity for test programs.
- do not compile and link libcurl sources to libtests _twice_ when libcurl
is built in static mode.
KNOWN ISSUES:
- running tests with unity builds may fail in cases.
- some build configurations/env may not compile in unity mode. E.g.:
https://ci.appveyor.com/project/curlorg/curl/builds/47230972/job/51wfesgnfuauwl8q#L250
Ref: https://github.com/libssh2/libssh2/issues/1034
Ref: https://cmake.org/cmake/help/latest/prop_tgt/UNITY_BUILD.html
Ref: https://en.wikipedia.org/wiki/Unity_buildCloses#11095
This is a common configuration that should be tested to avoid
regressions. The awsls cmake build was already out-of-tree so the
automake build now joins it.
Ref: #11006
- Makefile support for building test specific clients in tests/http/clients
- auto-make of clients when invoking pytest
- added test_09_02 for server PUSH_PROMISEs using clients/h2-serverpush
- added test_02_21 for lib based downloads and pausing/unpausing transfers
curl url parser:
- added internal method `curl_url_set_authority()` for setting the
authority part of a url (used for PUSH_PROMISE)
http2:
- made logging of PUSH_PROMISE handling nicer
Placing python test requirements in requirements.txt files
- separate files to base test suite and http tests since use
and module lists differ
- using the files in the gh workflows
websocket test cases, fixes for we and bufq
- bufq: account for spare chunks in space calculation
- bufq: reset chunks that are skipped empty
- ws: correctly encode frames with 126 bytes payload
- ws: update frame meta information on first call of collect
callback that fills user buffer
- test client ws-data: some test/reporting improvements
Closes#11006
- with `--proxy-http2` allow h2 ALPN negotiation to
forward proxies
- applies to http: requests against a https: proxy only,
as https: requests will auto-tunnel
- adding a HTTP/1 request parser in http1.c
- removed h2h3.c
- using new request parser in nghttp2 and all h3 backends
- adding test 2603 for request parser
- adding h2 proxy test cases to test_10_*
scorecard.py: request scoring accidentally always run curl
with '-v'. Removed that, expect double numbers.
labeller: added http1.* and h2-proxy sources to detection
Closes#10967
* Configure changes to detect AWS-LC
* CMakeLists.txt changes to detect AWS-LC
* Compile-time branches needed to support AWS-LC
* Correctly set OSSL_VERSION and report AWS-LC release number
* GitHub Actions script to build with autoconf and cmake against AWS-LC
AWS-LC is a BoringSSL/OpenSSL derivative
For more information see https://github.com/awslabs/aws-lc/Closes#10320
When a commit only contains tests, documentation, or cmake files, skip
those builds that aren't affected by those.
The file filters available on the CI services don't seem to allow
skipping individual jobs, only the entire workflow, so we can't get any
more fine-grained than this.
- added to: ngtcp2-quictls, ngtcp2-gnutls and the linux varians
quiche, bearssl, libressl, mbedtls, openssl3, rustls
- added disabled in ngtcp2-wolfssl due to weird SSL_connect() errors
not reproducable locally
Improvements on pytest:
- handling of systems with nghttpx in $PATH
- configure will seach $PATH got nghttpx used in pytest
- pytest fixes for managing nghttpx without h3 support
- ngtcp2-wolfssl: use a fully enabled wolfssl build
- lower parallel count for http/1.1 tests, since we do not
want to test excessive connections.
- check built curl for HTTPS-proxy support in proxy tests
- bearssl does not like one of our critical cert extensions, making
it non-critical now
- bearssl is too slow for test_12, skipping
- making sure we do h3 tests only when curl and server support is there
Closes#10699
Also skip builds on non-Windows platforms when only Windows build files
have changed.
This should reduce the number of useless builds and the associated
waiting time and chance of spurious failures, freeing resources for
new PRs.
Closes#10742
- httpd is only one server we test with
- the suite coveres the HTTP protocol in general where
the default test cases need a more beefy environment
Closes#10654
This reverts commit e0db842b2a.
This tool seems very restricted in how often it might be used by a
project and thus very quickly start to report fails simply because it
refuses to run when "there are more runs than allowed".
Closes#10613
- adding test_02_20 for reproducing the situation
- using recently released mod_h2 Apache module
- skipping test if an older version is installed
- adding installation of current mod_h2 to github pytest workflow
This reproduces the error reliable (for me) on the lib/http2.c version
of curl 7.88.0. And passes with the recent curl master.
Closes#10534
A temporary error with a remote server shouldn't cause a CI run to fail.
Also, put a cap on the time to download to fail faster on a misbehaving
server or connection and use HTTP compression where possible to reduce
download times.
New cfilter HTTP-CONNECT for h3/h2/http1.1 eyeballing.
- filter is installed when `--http3` in the tool is used (or
the equivalent CURLOPT_ done in the library)
- starts a QUIC/HTTP/3 connect right away. Should that not
succeed after 100ms (subject to change), a parallel attempt
is started for HTTP/2 and HTTP/1.1 via TCP
- both attempts are subject to IPv6/IPv4 eyeballing, same
as happens for other connections
- tie timeout to the ip-version HAPPY_EYEBALLS_TIMEOUT
- use a `soft` timeout at half the value. When the soft timeout
expires, the HTTPS-CONNECT filter checks if the QUIC filter
has received any data from the server. If not, it will start
the HTTP/2 attempt.
HTTP/3(ngtcp2) improvements.
- setting call_data in all cfilter calls similar to http/2 and vtls filters
for use in callback where no stream data is available.
- returning CURLE_PARTIAL_FILE for prematurely terminated transfers
- enabling pytest test_05 for h3
- shifting functionality to "connect" UDP sockets from ngtcp2
implementation into the udp socket cfilter. Because unconnected
UDP sockets are weird. For example they error when adding to a
pollset.
HTTP/3(quiche) improvements.
- fixed upload bug in quiche implementation, now passes 251 and pytest
- error codes on stream RESET
- improved debug logs
- handling of DRAIN during connect
- limiting pending event queue
HTTP/2 cfilter improvements.
- use LOG_CF macros for dynamic logging in debug build
- fix CURLcode on RST streams to be CURLE_PARTIAL_FILE
- enable pytest test_05 for h2
- fix upload pytests and improve parallel transfer performance.
GOAWAY handling for ngtcp2/quiche
- during connect, when the remote server refuses to accept new connections
and closes immediately (so the local conn goes into DRAIN phase), the
connection is torn down and a another attempt is made after a short grace
period.
This is the behaviour observed with nghttpx when we tell it to shut
down gracefully. Tested in pytest test_03_02.
TLS improvements
- ALPN selection for SSL/SSL-PROXY filters in one vtls set of functions, replaces
copy of logic in all tls backends.
- standardized the infof logging of offered ALPNs
- ALPN negotiated: have common function for all backends that sets alpn proprty
and connection related things based on the negotiated protocol (or lack thereof).
- new tests/tests-httpd/scorecard.py for testing h3/h2 protocol implementation.
Invoke:
python3 tests/tests-httpd/scorecard.py --help
for usage.
Improvements on gathering connect statistics and socket access.
- new CF_CTRL_CONN_REPORT_STATS cfilter control for having cfilters
report connection statistics. This is triggered when the connection
has completely connected.
- new void Curl_pgrsTimeWas(..) method to report a timer update with
a timestamp of when it happend. This allows for updating timers
"later", e.g. a connect statistic after full connectivity has been
reached.
- in case of HTTP eyeballing, the previous changes will update
statistics only from the filter chain that "won" the eyeballing.
- new cfilter query CF_QUERY_SOCKET for retrieving the socket used
by a filter chain.
Added methods Curl_conn_cf_get_socket() and Curl_conn_get_socket()
for convenient use of this query.
- Change VTLS backend to query their sub-filters for the socket when
checks during the handshake are made.
HTTP/3 documentation on how https eyeballing works.
TLS improvements
- ALPN selection for SSL/SSL-PROXY filters in one vtls set of functions, replaces
copy of logic in all tls backends.
- standardized the infof logging of offered ALPNs
- ALPN negotiated: have common function for all backends that sets alpn proprty
and connection related things based on the negotiated protocol (or lack thereof).
Scorecard with Caddy.
- configure can be run with `--with-test-caddy=path` to specify which caddy to use for testing
- tests/tests-httpd/scorecard.py now measures download speeds with caddy
pytest improvements
- adding Makfile to clean gen dir
- adding nghttpx rundir creation on start
- checking httpd version 2.4.55 for test_05 cases where it is needed. Skipping with message if too old.
- catch exception when checking for caddy existance on system.
Closes#10349
These owners do not have the bandwidth/energy to do the reviews which
makes PRs stall and this ownership claim flawed. We can bring people
back when the situation is different.
Follow-up to c04c78ac87Closes#10386
