Commit Graph

6045 Commits

Author SHA1 Message Date
Jan Venekamp
88727f7ed0
docs: improve cipher options documentation
Closes #14407
2024-08-17 11:14:21 +02:00
Stefan Eissing
a58b50fca6
transfer: Curl_sendrecv() and event related improvements
- Renames Curl_readwrite() to Curl_sendrecv() to reflect that it
  is mainly about talking to the server, not reads or writes to the
  client. Add a `nowp` parameter since the single caller already
  has this.
- Curl_sendrecv() now runs all possible operations whenever it is
  called and either it had been polling sockets or the 'select_bits'
  are set.
  POLL_IN/POLL_OUT are not always directly related to send/recv
  operations. Filters like HTTP/2, QUIC or TLS may monitor reverse
  directions. If a transfer does not want to send (KEEP_SEND), it
  will not do so, as before. Same for receives.
- Curl_update_timer() now checks the absolute timestamp of an expiry
  and the last/new timeout to determine if the application needs
  to stop/start/restart its timer. This fixes edge cases where
  updates did not happen as they should have.
- improved --test-event curl_easy_perform() simulation to handle
  situations where no sockets are registered but a timeout is
  in place.
- fixed bug in events_socket() that complained about removing
  a socket that was unknown, when indeed it had removed the socket
  just before, only it was the last in the list
- fixed conncache's internal handle to carry the multi instance
  (where the cache has one) so that operations on the closure handle
  trigger event callbacks correctly.
- fixed conncache to not POLL_REMOVE a socket twice when a conneciton
  was closed.

Closes #14561
2024-08-17 10:52:53 +02:00
Viktor Szakats
d8de4806e1
cmake: tidy-up continues
- move variable dump to a GHA foldable group.
- minimize scope for an include().
- rename `HIDES_CURL_PRIVATE_SYMBOLS` to `CURL_HIDES_PRIVATE_SYMBOLS`,
  to keep it in the curl namespace.
- drop quotes from a version number.
- add missing `Makefile.inc` var refs to comment.
- FindNGTCP2: rename internal var to underscore/lowercase.
- FindBearSSL, FindGSS: whitespace.

Closes #14571
2024-08-17 00:32:38 +02:00
Viktor Szakats
453d032b26
tidy-up: misc build, tests, lib/macos.c
- mkhelp.pl: drop unused assigment.
- Makefile.mk: update a comment.
- lib/macos.c: delete redundant block.
- fix two typos.

Closes #14558
2024-08-16 16:05:36 +02:00
Jan Venekamp
1e03d4bc0b
rustls: add support for setting TLS version and ciphers
Add support for CURLOPT_SSLVERSION, CURLOPT_TLS13_CIPHERS and
CURLOPT_SSL_CIPHER_LIST.

Closes #14535
2024-08-16 09:55:02 +02:00
Daniel Stenberg
dcb51bafab
splay: use access functions, add asserts, use Curl_timediff
- add set/get functions for the custom data in a tree node

- use Curl_timediff for time comparisons instead of the custom macro, as they
  do the exact same things.

- add asserts to catch mistakes better

- updated test 1309 accordingly

Closes #14562
2024-08-16 09:12:13 +02:00
Stefan Eissing
6c5a7af754
scorecard: tweak request measurements
Increase max-parallel up to 300, the curl max

Tweak output to just give the http response code

Closes #14564
2024-08-16 09:10:09 +02:00
Daniel Stenberg
3932964793
test1707: output diff more for debugging differences in CI outputs 2024-08-16 08:57:19 +02:00
Viktor Szakats
b042d5297d
tidy-up: misc spelling (bit, ASCII)
Closes #14559
2024-08-15 15:30:09 +02:00
Daniel Stenberg
551baf7d64
tests: move the disabling of 500 for hyper from CI to DISABLED
Follow-up to 136504195a

Closes #14551
2024-08-15 14:02:40 +02:00
Daniel Stenberg
5603204448
curl: fix the -w urle.* variables
urle.scheme, urle.user, urle.password and urle.options mistakenly
operated on the original URL instead of the *effective* (last) URL.

Add test 474 to verify.

Reported-by: Gruber Glass
Fixes #14550
Closes #14560
2024-08-15 14:01:24 +02:00
Viktor Szakats
9f23c8f201
cmake: fix and tidy up c-ares builds, enable in more CI jobs
- set `CARES_NO_DEPRECATED` to avoid c-ares deprecation warnings.
  Like autotools does.

- drop unused c-ares header directory when building libtests and test
  servers.

- disable TrackMemory (aka `CURLDEBUG`) feature for MSVC builds.
  It fails on test 558 and 1330, saying that TrackMemory isn't working.
  Left a FIXME about it.

- GHA/macos: enable c-ares in a cmake job.

- GHA/windows: enable c-ares in MSVC job.
  Fixes #14202

- GHA/windows: add c-ares mingw autotools job with tests.
  (move `--with-windows-unicode` option from 'default' job to
  this one to keep the former "default".)
  Put these tests on ignore for now:
  ```
  FAIL 472: 'aws-sigv4 with query' HTTP, aws-sigv4
  FAIL 1299: 'Send "OPTIONS *" with --request-target' HTTP, --request-target
  FAIL 1613: 'Send "OPTIONS *" with --request-target to a proxy' HTTP, HTTP proxy, --request-target
  ```
  Ref: https://github.com/curl/curl/actions/runs/10388126947/job/28765761441#step:15:5902

  472: fails with mingw despite Unicode enabled.
  1299, 1613: seem like a case of expanding '*' to a filename.

- GHA/windows: enable c-ares in mingw cmake job.

Closes #14541
2024-08-15 09:44:52 +02:00
Alex Snast
136504195a
getinfo: add CURLINFO_POSTTRANSFER_TIME_T
Returns the time, in microseconds, from the start until the last byte is
sent by libcurl (i.e. the request is sent off).

Closes #14189
2024-08-15 09:02:58 +02:00
Daniel Stenberg
c0233a35da
hash: provide asserts to verify API use
- converted the Curl_hash_count() macro to a function

- Discourage accessing struct fields directly

- Document the internal API in HASH.md

Closes #14503
2024-08-15 08:54:19 +02:00
Viktor Szakats
db39c668a8
cmake: sync up result variable names in Find modules
- bearssl, c-ares, gss, libpsl, libssh2, mbedtls:
  Before this patch these Find modules returned results via
  `<NAME>_INCLUDE_DIR` and `<NAME>_LIBRARY`.

  This patch makes them return `<NAME>_INCLUDE_DIRS` (note the `S`)
  and `<NAME>_LIBRARIES` like other modules already did.

- bearssl, mbedtls:
  Before this patch these Find modules allowed custom configuration
  via `<NAME>_INCLUDE_DIRS` (note the `S`).

  This patch makes them accept `<NAME>_INCLUDE_DIR`, like the rest of
  the modules did.

  Deprecate the old variables, but keep accepting them for
  compatibility.

- bearssl: add missing `mark_as_advanced()` call.

Closes #14542
2024-08-14 14:58:58 +02:00
Viktor Szakats
5abfe451be
cmake: update list of "advanced" variables
To hide them from the CMake GUI by default.

Closes #14540
2024-08-14 12:09:23 +02:00
Stefan Eissing
9222f31201
test649: improve robustness
Remove check of upload data before the error is triggered as this
may vary with network conditions.

Closes #14526
2024-08-14 11:50:02 +02:00
Stefan Eissing
e434cdb836
test587: improve robustness
Remove check of server output as upload may abort before request
could fully be sent, so server output may be completely missing.

Test already used a 1 second delay to mitigate timing. This change
makes timing no longer an issue.

Closes #14525
2024-08-14 11:49:05 +02:00
Stefan Eissing
68dad8c4ea
test httpd, tweak cipher list
Configure the AES 256 instead of the AES 128 cipher in the test httpd to
make scorecard testing between httpd and caddy more comparable.

Adapt test_17 expectations, now that AES 128 can no longer, but 256 can
now be negotiated.

Closes #14502
2024-08-14 11:48:35 +02:00
Viktor Szakats
1a444e3153
runtests: log ignored but passed tests
List tests explicitly ignored, but passing. It can be useful to catch
ignore rules that are no longer necessary because the tests are not
failing anymore.

To not clutter the default view, hide this information under a GitHub
Actions foldable group.

Closes #14457
2024-08-14 01:44:07 +02:00
Dan Fandrich
0c4f05c6ee tests: don't mangle output if hostname or type unknown
If either of these couldn't be determined, the output would be missing a
newline. Make sure the line ends even in the case of an error.
2024-08-13 01:13:00 -07:00
Viktor Szakats
af73743f86
curl_sha512_256: fix symbol collisions with nettle library
`SHA512_256_BLOCK_SIZE`, `SHA512_256_DIGEST_SIZE` macros were both
defined within curl and also in the nettle library required by GnuTLS.

Fix it by namespacing the curl macros.

Cherry-picked from #14495
Closes #14514
2024-08-13 10:04:06 +02:00
Viktor Szakats
624b20c637
lib: prefer CURL_SHA256_DIGEST_LENGTH over the unprefixed name
Already used in `vtls.h`. Prefer this curl-namespaced name over the
unprefixed `SHA256_DIGEST_LENGTH`. The latter is also defined by TLS
backends with a potential to cause issues.

Also stop relying on externel headers setting this constant. It's
already defined in `vtls.h` on curl's behalf, do this also for `lib`.

Cherry-picked from #14495
Closes #14513
2024-08-13 10:04:06 +02:00
Viktor Szakats
457427e03f
build: silence C4232 MSVC warnings in vcpkg ngtcp2 builds
Silence bogus MSVC warning C4232. Use the method already used
for similar cases earlier.

Also fixup existing suppressions to use pragma push/pop.

```
lib\vquic\curl_ngtcp2.c(709,40): error C2220: the following warning is treated as an error
lib\vquic\curl_ngtcp2.c(709,40): warning C4232: nonstandard extension used: 'client_initial': address of dllimport 'ngtcp2_crypto_client_initial_cb' is not static, identity not guaranteed
lib\vquic\curl_ngtcp2.c(709,40): warning C4232: nonstandard extension used: 'recv_crypto_data': address of dllimport 'ngtcp2_crypto_recv_crypto_data_cb' is not static, identity not guaran
lib\vquic\curl_ngtcp2.c(709,40): warning C4232: nonstandard extension used: 'encrypt': address of dllimport 'ngtcp2_crypto_encrypt_cb' is not static, identity not guaranteed
lib\vquic\curl_ngtcp2.c(709,40): warning C4232: nonstandard extension used: 'decrypt': address of dllimport 'ngtcp2_crypto_decrypt_cb' is not static, identity not guaranteed
lib\vquic\curl_ngtcp2.c(709,40): warning C4232: nonstandard extension used: 'hp_mask': address of dllimport 'ngtcp2_crypto_hp_mask_cb' is not static, identity not guaranteed
lib\vquic\curl_ngtcp2.c(709,40): warning C4232: nonstandard extension used: 'recv_retry': address of dllimport 'ngtcp2_crypto_recv_retry_cb' is not static, identity not guaranteed
lib\vquic\curl_ngtcp2.c(709,40): warning C4232: nonstandard extension used: 'update_key': address of dllimport 'ngtcp2_crypto_update_key_cb' is not static, identity not guaranteed
lib\vquic\curl_ngtcp2.c(709,40): warning C4232: nonstandard extension used: 'delete_crypto_aead_ctx': address of dllimport 'ngtcp2_crypto_delete_crypto_aead_ctx_cb' is not static, identit
lib\vquic\curl_ngtcp2.c(709,40): warning C4232: nonstandard extension used: 'delete_crypto_cipher_ctx': address of dllimport 'ngtcp2_crypto_delete_crypto_cipher_ctx_cb' is not static, ide
lib\vquic\curl_ngtcp2.c(709,40): warning C4232: nonstandard extension used: 'get_path_challenge_data': address of dllimport 'ngtcp2_crypto_get_path_challenge_data_cb' is not static, ident
```
Ref: https://github.com/curl/curl/actions/runs/10343459009/job/28627621355#step:10:30

Cherry-picked from #14495
Co-authored-by: Tal Regev
Ref: #14383
Closes #14510
2024-08-13 09:28:27 +02:00
Viktor Szakats
fdc3e88bf0
IDN: fix/extend/migrate test exclusion rules
- make sure to exclude failing tests when libidn2 is detected by
  default.

- ignore test 1560 results. Seen to fail with libidn2.
  I'm not sure why this test was not executed earlier:
  https://github.com/curl/curl/actions/runs/10354610889/job/28660309355#step:13:3647

- runtests: recognize `libidn2` as a feature.

- move IDN test exclusions from GHA/windows to `tests/data/DISABLED`.

- GHA/windows: drop default `-DUSE_LIBIDN2=ON` cmake config.

Cherry-picked from #14495
Closes #14519
2024-08-13 09:28:26 +02:00
Jan Venekamp
902d9a1d4e
wolfssl: fix CURLOPT_SSLVERSION
Before, setting CURLOPT_SSLVERSION with wolfSSL restricted the the tls
proto to just the specified version. Now it properly supports a range.
So it can set the min and max tls proto (max requires wolfSSL 4.2.0).

Bump the absolute minimum required version of wolfSSL to 3.4.6 (released
2015) because it is needed for the wolfSSL_CTX_SetMinVersion() function.

Closes #14480
2024-08-12 19:20:28 +02:00
Viktor Szakats
d222dbe788
build: tidy up internal macro names for libcurl.pc
Rename internal macros to match their `libcurl.pc` metadata counterpart.
Also apply these to the `curl-config.in` template.

- `CPPFLAG_CURL_STATICLIB` -> `LIBCURL_PC_CFLAGS`
- `LIBCURL_LIBS`           -> `LIBCURL_PC_LIBS_PRIVATE`
- `LIBCURL_NO_SHARED`      -> `LIBCURL_PC_LIBS`

Closes #14476
2024-08-12 14:56:41 +02:00
Viktor Szakats
f3b14e1b0c
tidy-up: delete Makefile.inc from EXTRA_DIST
autotools is adding them automatically. Delete the few ones that were
also added manually.

Closes #14496
2024-08-12 14:54:47 +02:00
Daniel Stenberg
ba235ab269
llist: remove direct struct accesses, use only functions
- Turned them all into functions to also do asserts etc.

- The llist related structs got all their fields renamed in order to make
  sure no existing code remains using direct access.

- Each list node struct now points back to the list it "lives in", so
  Curl_node_remove() no longer needs the list pointer.

- Rename the node struct and some of the access functions.

- Added lots of ASSERTs to verify API being used correctly

- Fix some cases of API misuse

Add docs/LLIST.md documenting the internal linked list API.

Closes #14485
2024-08-12 13:18:10 +02:00
Viktor Szakats
aa3a31ce26
cmake: exclude tests/http/clients builds by default
To limit building them with the testdeps target, like it's done with
the rest of test programs.

Follow-up to 232302f88a #14382
Closes #14477
2024-08-10 10:02:48 +02:00
Viktor Szakats
a298df7f47
cmake: more small tidy-ups
- prefix local variables with underscore and convert to lowercase.
- list variables accepted by `libcurl.pc` and `curl-config` templates.
- quote more string literals.

Follow-up to 919394ee64 #14450
Closes #14462
2024-08-09 10:45:40 +02:00
Stefan Eissing
f27ba323ad
test677: improve robustness
Do not treat CURLE_AGAIN as error.

Fixes test with CURL_DBG_SOCK_WBLOCK=90 set.

Closes #14455
2024-08-09 09:39:59 +02:00
Stefan Eissing
640febc7d0
test579: improve robustness
Log progress only at start and end of transfer to give normalized
output when upload data is only partially sent or temporarily blocked.

Fixes test with CURL_DBG_SOCK_WBLOCK=90 set.

Closes #14454
2024-08-09 09:39:17 +02:00
Stefan Eissing
ac6349b451
test556: improve robustness
Fix handling of partial/blocked sends.

Fixes test with CURL_DBG_SOCK_WBLOCK=90 set.

Closes #14453
2024-08-09 09:38:19 +02:00
Viktor Szakats
919394ee64
cmake: more small tidy-ups and fixes
- tidy up two `MATCHES` expression by avoiding macros expansion and
  adding quotes. Then convert then to `STREQUAL` to match other places
  in the code doing the same checks.

- fix setting `_ALL_SOURCE` for AIX to match what autotools does.

- delete stray `_ALL_SOURCE` reference from `lib/config_riscos.h`

- simplify/fix two `STREQUAL ""` checks.
  The one in the `openssl_check_symbol_exists()` macro succeeded
  regardless of the value. The other could return TRUE when
  `CMAKE_OSX_SYSROOT` was undefined.

- delete code for CMake versions (<3.7) we no longer support.

- prefer `LIST(APPEND ...)` to extend `CURL_LIBS`.

- use `CURL_LIBS` to add the `network` lib for Haiku.
  Before this patch it was done via raw C flags. I could not test this.

- move `_WIN32_WINNT`-related code next to each other.
  It also moves detection to the top, allowing more code to use
  the result.

- merge two `WIN32` blocks.

- rename internal variables to underscore + lowercase.

- unwrap a line, indent, whitespace.

Closes #14450
2024-08-08 13:48:28 +02:00
Viktor Szakats
c2889a7b41
cmake: more syntax tidy-up
- quote string literals.
  In the hope it improves syntax-highlighting and readability.

- use lowercase, underscore-prefixed local var names.
  As a hint for scope, to help readability.

- prefer `pkg_search_module` (over `pkg_check_modules`).
  They are the same, but `pkg_search_module` stops searching
  at the first hit.

- more `IN LISTS` in `foreach()`.

- OtherTests.cmake: clear `CMAKE_EXTRA_INCLUDE_FILES` after use.

- add `PROJECT_LABEL` for http/client and unit test targets.

- sync `Find*` module comments and formatting.

- drop a few local variables.

- drop bogus `CARES_LIBRARIES` from comment.

- unquote numeric literal.

Follow-up to acbc6b703f #14197
Closes #14388
2024-08-07 23:41:27 +02:00
Stefan Eissing
0bc5b2e37c
http/2: simplify eos/blocked handling
- rely on the new flush to handle blocked sends. No longer
  do simulated EAGAIN on (partially) blocked sends with their
  need to handle repeats.
- fix some debug handling CURL_SMALLREQSEND env var
- add some assertings in request.c for affirming we do it right
- enhance assertion output in test_16 for easier analysis

Closes #14435
2024-08-07 18:28:49 +02:00
Bo Anderson
588a6e334a
idn: more strictly check AppleIDN errors
UIDNA API returns two error values but we were only checking one.
Checking both better aligns the behaviour with that of libidn2.

Closes #14431
2024-08-07 14:59:07 +02:00
Bo Anderson
a35687831f
idn: support non-UTF-8 input under AppleIDN
This aligns the behaviour with libidn2 and the curl documentation.

Closes #14431
2024-08-07 14:59:01 +02:00
Viktor Szakats
493c6d79eb
cmake: delete MSVC warning suppression for tests/server
Server code no longer produces this warning.

Closes #14428
2024-08-07 11:35:16 +02:00
Viktor Szakats
b1153820f4
dist: add missing test_*.py scripts
Closes #14427
2024-08-07 11:35:16 +02:00
Dan Fandrich
5f9426ec43 tests: show snapshot commit in testcurl
This disambiguates the source code being tested. The output format is
the same as when testing out of a git repo, but with no description and
a long hash.

Ref: #14363
Closes #14429
2024-08-07 00:22:38 -07:00
Stefan Eissing
b102763c19
curl: fix --test-event --parallel
(in debug-builds)

Fix implementation in curl using libuv to process parallel transfers.
Add pytest capabilities to run test cases with --test-event.

- fix uv_timer handling to carry correct 'data' pointing to uv context.
- fix uv_loop handling to reap and add transfers when possible
- fix return code when a transfer errored

Closes #14413
2024-08-07 08:57:05 +02:00
Daniel Stenberg
cd4aee156f
tests: ignore QUIT from FTP protocol comparisons
- QUIT is not an important FTP command
- curl only sends it "best effort", meaning it might not be sent
- it is a known "flaky" thing in test output because of this

Closes #14404
2024-08-07 08:48:56 +02:00
Daniel Stenberg
48818a41af
tests/runner: only allow [!A-Za-z0-9_-] in %if feature names
... in the %if preprocessor parser.

This guards against accidentally getting a carriage-return part of the
feature name.

Fixes #14403
Closes #14411
2024-08-07 00:30:17 +02:00
Daniel Stenberg
b0394b1535
runtests: if DISABLED cannot be read, error out
Because then runtests cannot disable the right set of tests and things
will be annoying.

Closes #14411
2024-08-07 00:30:17 +02:00
Viktor Szakats
c6fb9895b6
cmake: cleanup header paths
- sync build-dir/source-dir header path order with autotools, by
  including build-dir first, then source-dir.
  This prevents out-of-tree builds breaking due to leftover generated
  headers in the source tree.

- tests/unit: move `src` ahead of `libtest` in header path, syncing with
  autotools.

- stop adding non-existing generated `include` dir to header path.
  There are no generated `include` headers and this directory is either
  missing in out-of-tree builds or the same as the one already added
  globally via the root `CMakeLists.txt`.

- lib: stop adding a duplicate source include directory to the header
  path.
  It's already added globally via the root `CMakeLists.txt`.

- lib: stop adding the project root to the header path.

- docs/examples: drop internal header paths.
  Examples do not and should not use internal headers.

- replace `curl_setup_once.h` in comments with `curl_setup.h`,
  the header actually used, and also referred to in autotools comments.

- add comment why we need `src` in include path for `tests/server`.

- add quotes around header directories.

Closes #14416
2024-08-06 22:33:05 +02:00
Daniel Stenberg
dcc5209520
GHA: run badwords check on tests/*.md too
With several wording updates accordingly.

Closes #14412
2024-08-06 13:43:25 +02:00
Daniel Stenberg
91fcbc5d1a
dist: drop buildconf
The documented and mandated step has been to not use buildconf but to
invoke 'autoreconf -fi' for four years already.

This change only drops buildconf from the release tarball, it remains
present in git for now.

Follow-up to 85868537d6

Closes #14412
2024-08-06 13:43:20 +02:00
Viktor Szakats
232302f88a
cmake: add Linux CI job, fix pytest with cmake
- extend existing Linux workflow with CMake support.
  Including running pytest the first time with CMake.

- cmake: generate `tests/config` and `tests/http/config.ini`.
  Required for pytest tests.
  Uses basic detection logic. Feel free to take it from here.
  Also dump config files in a CI step for debugging purposes.

- cmake: build `tests/http/clients` programs.

- fix portability issues with `tests/http/clients` programs.
  Some of them use `getopt()`, which is not supported by MSVC.
  Fix the rest to compile in CI (old-mingw-w64, MSVC, Windows).

- GHA/linux: add CMake job matching an existing autotools one.

- GHA/linux: test `-DCURL_LIBCURL_VERSIONED_SYMBOLS=ON`
  in the new CMake job.

- reorder testdeps to build server, client tests first and then
  libtests and units, to catch errors in the more complex/unique
  sources earlier.

- sort list in `tests/http/clients/Makefile.inc`.

Closes #14382
2024-08-06 02:40:32 +02:00