Commit Graph

33666 Commits

Author SHA1 Message Date
Daniel Stenberg
59831f8061
tool_operate: split up the huge single_transfer into sub functions
- split up in a few smaller and easier to read functions
- simplify several sections
- avoid superfluous extra allocations
- remove unused debug code

Closes #15385
2024-10-23 23:12:54 +02:00
Daniel Stenberg
30da1f5974
setopt: split Curl_vsetopt() into several sub functions
Reduce the ~3000 line super function into smaller pieces, easier to read and
manage.

Extract the option's argument earlier and use a fixed type instead of using
va_arg() everywhere.

Closes #15376
2024-10-23 23:09:48 +02:00
Viktor Szakats
b3816f67b9
cmake: avoid setting BUILD_TESTING
`BUILD_TESTING` variable is used by other projects and CMake internally.
Replace `cmake_dependent_option()` with `option()` and introduce an
internal variable to track if want and can do testing.

Follow-up to #6036
Follow-up to 3a1e798009 #6072

Reported-by: Robert Maynard
Fixes #15351
Closes #15355
2024-10-23 21:24:02 +02:00
Viktor Szakats
7c023c3f6e
libssh2: delete duplicate break
```
lib/vssh/libssh2.c:2495:7: warning: 'break' will never be executed [-Wunreachable-code-break]
      break;
      ^~~~~
```

CI did not catch it due to llvm skipping this check for all #included
files. It's designed this way to avoid performance issues and false
positive when checking headers:
https://github.com/llvm/llvm-project/issues/71046

Closes #15384
2024-10-23 19:36:04 +02:00
Viktor Szakats
6b440704d8
GHA: drop "3" from openssl names and keys
Also:
- drop patch suffix from cache key for thread-sanitizer local build
  Follow-up to 73d2779196 #15379

Closes #15383
2024-10-23 19:36:03 +02:00
Viktor Szakats
b8de0dadd9
cmake: tidy up line order [ci skip] 2024-10-23 13:13:01 +02:00
Viktor Szakats
5f9411f953
GHA/windows: work around Git for Windows perf regression
Fix the significant perf regression for vcpkg jobs by switching to the
MSYS2 shell environment from Git for Windows. This env is already used
for old-mingw-w64 job that remained unaffected by this issue.

The issue began with the windows-runner update 20241015.1.0. It bumped
Git for Windows from Git 2.46.2.windows.1 to Git 2.47.0.windows.1. GfW
bumped its MSYS2 components, including `msys-2.0.dll`. That's Cygwin
code, which may have contributed to this. Pipes were involved and
`runtests.pl` relies on pipes heavily in parallel mode. (The issue was
not seen with parallel tests disabled, in retrospect.)

This is useful as a permanent solution too. It drop GfW as a dependency
and makes Windows jobs use one less shell/env flavour.

Long term it might help to use native Windows Perl to avoid the MSYS
layer completely, if there is a way to make that work.

Assortment of possibly related links:
https://cygwin.com/pipermail/cygwin/2024-August/256398.html
f78009cb1c
7f3c225325

https://github.com/actions/runner-images/issues/10843
https://github.com/git-for-windows/git/issues/5199
https://github.com/git-for-windows/msys2-runtime/pull/75
7913a41703
555afcb2f3
1c5f4dcdc5

Follow-up to c33174d42f #15364
Follow-up to 1e0305973c #15356

Closes #15380
2024-10-23 13:00:41 +02:00
Viktor Szakats
73d2779196
GHA/linux: drop patch from openssl3 thread sanitizer
The patch is now part of the 3.4.0 stable release.
(Turns out it was part of 3.3.2 already.)

Also:
- rename this local build to match the scheme used with wolfssl.
- drop '3' from local openssl build name.
- sync job name with others.
- quote step names where missing.

Follow-up to a2bcec0ee0 #14751
Closes #15379
2024-10-23 12:02:17 +02:00
renovate[bot]
e1099726a5
CI: update dependency openssl/openssl to v3.4.0
Closes #15377
2024-10-23 11:24:21 +02:00
Viktor Szakats
53fdc5faf3
runtests: use deterministic sort for TESTINFO lines
Sort TESTINFO lines by description within the number of skipped test.
It makes the list of skipped test groups easier to diff/compare between
jobs and runs.

Closes #15374
2024-10-23 10:42:30 +02:00
Max Dymond
e43d37c541
ci: fix renovate's matching for OpenSSL and quictls
Renovate only matches on the raw version numbers of a package, but
OpenSSL includes `openssl-` as a prefix in the version number. This
change means that the match string now expects the `openssl-` prefix
and will just update the version portion.

This also updates quictls so that renovate can detect and update the
version correctly.

Closes #15359
2024-10-23 10:40:43 +02:00
Viktor Szakats
b327a53f09
GHA: use --no-install-suggests --no-install-recommends where missing
It prevents `apt-get install` commands installing unnecessary packages.

Makes the 8 HTTP/3 jobs around 30 seconds faster each.

before: https://github.com/curl/curl/actions/runs/11466168597
after:  https://github.com/curl/curl/actions/runs/11469013245?pr=15373

Closes #15373
2024-10-23 01:29:52 +02:00
Daniel Stenberg
0e18bd3940
mk-lib1521: fix the long return code check
It worked mostly by accident since it checked the variable from the
previous setopt invoke.

Closes #15372
2024-10-23 00:41:05 +02:00
Viktor Szakats
605bc2d2c6
GHA/linux: merge 32-bit Linux workflow
Also:
- tidy up a step condition.

Closes #15370
2024-10-22 23:38:13 +02:00
Yedaya Katsman
acd134cfe1 tests: Fix FILEFORMAT <file name=""> directive
Follow-up to dcc52095

Closes #15371
2024-10-22 13:41:02 -07:00
Viktor Szakats
b6219cd932
GHA/linux: merge torture jobs into the main workflow
They complete in 4 and 7 minutes, and do not hold back the main Linux
workflow.

Also:
- bump default parallelism for `test-torture` target to `-j20`
  (was: `-j2`).
- drop redundant package install from `rustls` jobs.

Closes #15360
2024-10-22 20:20:42 +02:00
Viktor Szakats
52851d3256
GHA/macos: use test-torture target for torture tests
They used `test-ci` before this patch.

Closes #15369
2024-10-22 20:20:42 +02:00
Viktor Szakats
096bfdeff6
cmake/FindCares: fix version detection for c-ares 1.34.1
Due to a regression in c-ares 1.34.1, the non-pkg-config version
detection method broke for this version. c-ares 1.34.2 fixes it,
but update our detection code anyway to also work with 1.34.1.

Ref:
126e274159
https://github.com/c-ares/c-ares/pull/903

Closes #15368
2024-10-22 20:20:42 +02:00
Viktor Szakats
825a800e0e
cmake: use the BSD variable
- use `BSD` in addition to backwards-compatible method.
- add `BSD` to the configuration log and `buildinfo.txt` if detected.
- add `BSD` tag to `buildinfo.txt` also via `./configure`.

The `BSD` variable is supported by CMake 3.25.0 and upper.

Ref: https://cmake.org/cmake/help/latest/variable/BSD.html

Closes #15367
2024-10-22 20:20:41 +02:00
Viktor Szakats
9126eb5a8a
cmake: replace CURL_*_DIR with {PROJECT,CMAKE_CURRENT}_*_DIR
It reduces the number of synonym variables in the code.
Makes it easier to grok and grep.

- replace `CURL_SOURCE_DIR`
  with `PROJECT_SOURCE_DIR`.

- replace `CURL_BINARY_DIR`
  with `PROJECT_BINARY_DIR` or `CMAKE_CURRENT_BINARY_DIR`.

- replace a single use of `CMAKE_BINARY_DIR`
  with `PROJECT_BINARY_DIR`.

- replace `CMAKE_CURRENT_*_DIR`
  with `PROJECT_*_DIR` where it makes the code more uniform.

- quote an argument (formatting).

Closes #15331
2024-10-22 19:13:08 +02:00
Viktor Szakats
c33174d42f
GHA/windows: increase timeout for vcpkg jobs due to slowness
The openssl job no longer fits into 10 minutes since the 20241015.1.0
GHA windows-latest image update. This caused all runs to fail.

The `run tests` step takes ~10 minutes now, up from ~4. This is
6 minutes more than before these updates. It's seen with other vcpkg
jobs too, tests run slower than half speed since.

Bump the timeout to make it, though the headroom is now less than it
was.

Before:
https://github.com/curl/curl/actions/runs/11386748199/job/31679733295
https://github.com/curl/curl/actions/runs/11347976608/job/31560690219

After:
https://github.com/curl/curl/actions/runs/11462332743/job/31893491625?pr=15364

Ref: fcc4cdb1d0
Ref: https://github.com/curl/curl/pull/15335#issuecomment-2423759953
Follow-up to 1e0305973c #15356
Closes #15364
2024-10-22 19:13:08 +02:00
Daniel Stenberg
dcb27fdd40
GHA: fix the msh3 renovate thing
Follow-up to 943df95ae7

Closes #15363
2024-10-22 15:48:27 +02:00
Stefan Eissing
943df95ae7
CI: run with standard mod_http2
We used to include a special mod_h2 in our CI that supports the
directive H2MaxDataFrameLen for test_02_20. Since then, ubuntu-lastest
includes a more recent apache httpd. Let's see if we can live without
the special.

Closes #15353
2024-10-22 15:31:46 +02:00
Tal Regev
c2e2636773
GHA/windows: add http3 to libressl vcpkg job
Also drop a libressl workaround no longer necessary.

Closes #15338
2024-10-22 14:23:12 +02:00
Viktor Szakats
1e0305973c
GHA/windows: ignore results for test 987
987 is `SMTPS with redundant explicit SSL request`.

Root cause undiscovered.

Started failing after GHA bumping the windows image to `20241015.1.0`:
fcc4cdb1d0
https://github.com/actions/runner-images/pull/10803

vcpkg packages also got bumped as a result. They seem unrelated:
c-ares  1.33.1 -> 1.34.1
nghttp2 1.62.1 -> 1.63.0
nghttp3 1.5.0  -> 1.6.0
(there may be more)

Ref: https://github.com/curl/curl/pull/15335#issuecomment-2423759953

Closes #15356
2024-10-22 14:17:58 +02:00
Viktor Szakats
0978afd7aa
GHA/linux: tidy up and performance
- replace openssl3 default local build with packaged one.
- drop valgrind from IntelC job.
- drop IntelC no-ssl job.
- bump local openssl to 3.3.2.
- disable tests in the cmake variant of a job.
- add comment to the remaining local openssl3 build.
  We can drop the patch after upgrading to upcoming 3.4.0.
- drop gcc-11 from jobs. packaged gcc is now newer at 13.2.0.
  (saves more than 1m install time for each of the 5 jobs.)
  Follow-up to 9cc9a6472c #9454

Closes #15349
2024-10-22 13:15:37 +02:00
zjyhjqs
e89491e1f0
cmake: fix compile warnings for clang-cl
clang-cl is an alternative command-line interface to Clang, designed
for compatibility with the Visual C++ compiler, `cl.exe`:
https://clang.llvm.org/docs/UsersManual.html#clang-cl

The way to test clang-cl in CMake:
- `CMAKE_<LANGUAGE>_COMPILER_ID`: "Clang"
- `CMAKE_<LANGUAGE>_COMPILER_FRONTEND_VARIANT`: "MSVC"

Note: `CMAKE_<LANGUAGE>_COMPILER_FRONTEND_VARIANT` was introduced since
CMake 3.14, but the variable `MSVC` works fine here.

https://cmake.org/cmake/help/latest/variable/CMAKE_LANG_COMPILER_FRONTEND_VARIANT.html
https://cmake.org/cmake/help/latest/variable/MSVC.html

Closes #15337
2024-10-21 12:47:47 +02:00
Viktor Szakats
7dd7cbac88
version: say quictls in MSH3 builds
Before: `curl 8.11.0-DEV (x86_64-pc-linux-gnu) libcurl/8.11.0-DEV OpenSSL/3.3.0 zlib/1.3 brotli/1.1.0 zstd/1.5.5 libpsl/0.21.2 msh3/0.6.0.0`
After:  `curl 8.11.0-DEV (x86_64-pc-linux-gnu) libcurl/8.11.0-DEV quictls/3.3.0 zlib/1.3 brotli/1.1.0 zstd/1.5.5 libpsl/0.21.2 msh3/0.6.0.0`

Closes #15335
2024-10-21 12:47:46 +02:00
Gabriel Marin
a58584a881
checksrc: add check for spaces around logical AND operators
Closes #15144
2024-10-21 09:35:00 +02:00
Daniel Stenberg
51724c43e3
curl_ws_recv.md: the 'meta' pointer is only returned on success
Reported-by: Dylam De La Torre
Fixes #15340
Closes #15343
2024-10-20 22:58:27 +02:00
Daniel Stenberg
d6bae1cb8f
curl_ws_recv: return recv 0 and point meta to NULL on all errors
Previously it could accidentally return some errors without having reset
the values.

Closes #15342
2024-10-20 22:57:24 +02:00
Viktor Szakats
2816cba2d7
GHA/linux: bump to quictls 3.3.0
Closes #15334
2024-10-20 01:16:28 +02:00
Jay Satiro
547d600476 curl_multi_perform.md: fix typo
Reported-by: Dylam De La Torre

Fixes https://github.com/curl/curl/issues/15339
2024-10-19 12:34:27 -04:00
Dan Fandrich
6847733191 docs: fix a typo in some cipher options 2024-10-18 14:42:34 -07:00
renovate[bot]
e29629a402
GHA: update ngtcp2/ngtcp2 and awslabs/aws-lc
- update ngtcp2/ngtcp2 to v1.8.1
- update awslabs/aws-lc to v1.37.0

Closes #15318
Closes #15329
2024-10-18 16:43:52 +02:00
renovate[bot]
2c1d83e6a7
Dockerfile: update Docker digest to d830561
Closes #15315
2024-10-18 16:39:26 +02:00
Viktor Szakats
4de627ab09
winbuild: add initial wolfSSL support
Ref: https://datagirl.xyz/posts/wolfssl_curl_w2k.html

Closes #15264
2024-10-18 12:58:50 +02:00
Daniel Stenberg
b4e162566f
KNOWN_BUGS: LDFLAGS passed too late
Makes linking fail on some (ancient) platforms.

Closes #14893
Closes #15306
2024-10-18 11:52:30 +02:00
Daniel Stenberg
5ea61a0b54
hsts: support "implied LWS" properly around max-age
Adjust test 780 to verify.

Reported-by: newfunction
Closes #15330
2024-10-18 11:42:42 +02:00
Daniel Stenberg
288cfcbe38
RELEASE-NOTES: synced 2024-10-18 10:13:38 +02:00
zjyhjqs
fbc0da376f
cmake: set version for project() and add CPack support
Note: the version like `8.11.0-DEV` is not a valid version for
`project()`, so need to extract the major, minor and patch parts.

Previous, manual, `CURL_VERSION` macro is defined by `project()`
after this patch, so rename existing `CURL_VERSION*` variables to
`_curl_version*`.

Closes #15281
2024-10-18 10:00:29 +02:00
Daniel Stenberg
1b155f034a
tool_operate: reuse the schannel backend check
The transfer_per_config is called once per new transfer. It now saves
the result of the first TLS backend check done so that subsequent
invokes are more efficient and reuses the existing knowledge.

This change also splits the logic into several smaller functions.

Closes #15323
2024-10-18 08:13:53 +02:00
Daniel Stenberg
29faa7919b
libcurl/opts: improve phrasing for connection cap related options
Unify, clarify.

Closes #15324
2024-10-17 23:06:33 +02:00
Stefan Eissing
fe2a72029e
http2: auto reset stream on server eos
When a server signals EOS from its side and the curl upload is
unfinished and the server has not given a positive HTTP status response,
auto RST the stream to signal that the upload is incomplete and that the
whole transfer can be stopped.

Fixes the case where the server responds with 413 on an upload but does
not RST the stream from its side, as httpd and others do.

Reported-by: jkamp-aws on github
Fixes #15316
Closes #15325
2024-10-17 23:03:41 +02:00
Daniel Stenberg
2ae8d9b579
libtests: generate the lib1521 atomically
By renaming from a temporary file name to the .c once completed. This
avoids the risk that the checksrc job tries to verify the file before it
is complete, in parallel build setups.

Reported-by: Dan Frandrich
Fixes #15258
Closes #15327
2024-10-17 23:00:24 +02:00
Daniel Stenberg
b9877b74c5
GHA: drop the hyper job
Hyper support is being removed in 2025. No one works on it. Getting
flaky test runs with this job adds nothing to the project.

Closes #15326
2024-10-17 22:56:45 +02:00
Stefan Eissing
b42eb27c1f
openssl: improve retries on shutdown
Once SSL_shutdown() has been called, OpenSSL does not really seem to
like it when it is called again and the other side has some finally data
to deliver.

Instead SSL_read() needs to be used solely, once the close notify has
been sent from curl's side.

Closes #15321
2024-10-17 16:11:06 +02:00
Daniel Stenberg
8cdbaba4bf
tool_operate: break out of loop on error
Follow-up to 69bf530dfd

The loop could get stuck there in torture tests/OOM.

Closes #15322
2024-10-17 16:04:09 +02:00
Daniel Stenberg
38bfe1c2aa
GHA: switch off proselint
Because we cannot disable the individual warnings we do not care about,
making this tool almost unusable for our purposes. See
https://github.com/amperser/proselint/issues/1367

Instead, make 'very' a banned word (as recently that has been what
proselint most commonly points out for us).

Closes #15314
2024-10-17 13:36:42 +02:00
Daniel Stenberg
9cc246401e
source: avoid use of 'very' in comments 2024-10-17 13:36:33 +02:00