Commit Graph

30960 Commits

Author SHA1 Message Date
Stefan Eissing
944e219f10
http2: upgrade tests and add fix for non-existing stream
- check in h2 filter recv that stream actually exists
  and return error if not
- add test for parallel, extreme h2 upgrades that fail if
  connections get reused before fully switched
- add h2 upgrade upload test just for completeness

Closes #11563
2023-08-03 17:05:13 +02:00
Viktor Szakats
db70846e2e
tests: ensure libcurl.def contains all exports
Add `test1279` to verify that `libcurl.def` lists all exported API
functions found in libcurl headers.

Also:

- extend test suite XML `stdout` tag with the `loadfile` attribute.

- fix `tests/extern-scan.pl` and `test1135` to include websocket API.

- use all headers (sorted) in `test1135` instead of a manual list.

- add options `--sort`, `--heading=` to `tests/extern-scan.pl`.

- add `libcurl.def` to the auto-labeler GHA task.

Follow-up to 2ebc74c36a

Closes #11570
2023-08-03 11:07:47 +00:00
Daniel Stenberg
d135d040df
url: change default value for CURLOPT_MAXREDIRS to 30
It was previously unlimited by default, but that's not a sensible
default. While changing this has a remote risk of breaking an existing
use case, I figure it is more likely to actually save users from loops.

Closes #11581
2023-08-02 23:59:59 +02:00
Daniel Stenberg
367a66b130
lib: fix a few *printf() flag mistakes
Reported-by: Gisle Vanem
Ref: #11574
Closes #11579
2023-08-02 23:11:09 +02:00
Samuel Chiang
20f4e94eeb
openssl: make aws-lc version support OCSP
And bump version in CI

Closes #11568
2023-08-02 16:23:45 +02:00
Daniel Stenberg
ce1fc742c1
tool: make the length argument an int for printf()-.* flags
Closes #11578
2023-08-02 16:11:55 +02:00
Daniel Stenberg
139ecfb869
tool_operate: fix memory leak when SSL_CERT_DIR is used
Detected by Coverity

Follow-up to 29bce9857a

Closes #11577
2023-08-02 16:10:49 +02:00
Daniel Stenberg
c17bf311f1
tool/var: free memory on OOM
Coverity detected this memory leak in OOM situation

Follow-up to 2e160c9c65

Closes #11575
2023-08-02 16:09:43 +02:00
Viktor Szakats
ddb179586f
gha: bump libressl and mbedtls versions
Closes #11573
2023-08-02 10:39:01 +00:00
Jay Satiro
b4f9ae5126 schannel: fix user-set legacy algorithms in Windows 10 & 11
- If the user set a legacy algorithm list (CURLOPT_SSL_CIPHER_LIST) then
  use the SCHANNEL_CRED legacy structure to pass the list to Schannel.

- If the user set both a legacy algorithm list and a TLS 1.3 cipher list
  then abort.

Although MS doesn't document it, Schannel will not negotiate TLS 1.3
when SCHANNEL_CRED is used. That means setting a legacy algorithm list
limits the user to earlier versions of TLS.

Prior to this change, since 8beff435 (precedes 7.85.0), libcurl would
ignore legacy algorithms in Windows 10 1809 and later.

Reported-by: zhihaoy@users.noreply.github.com

Fixes https://github.com/curl/curl/pull/10741
Closes https://github.com/curl/curl/pull/10746
2023-08-02 03:43:13 -04:00
Daniel Stenberg
4169943d15
variable.d: setting a variable again overwrites it
Reported-by: Niall McGee
Bug: https://twitter.com/niallmcgee/status/1686523075423322113
Closes #11571
2023-08-02 09:32:08 +02:00
Jay Satiro
783e4714d9 CURLOPT_PROXY_SSL_OPTIONS.3: sync formatting
- Re-wrap CURLSSLOPT_ALLOW_BEAST description.
2023-08-02 03:19:29 -04:00
Daniel Stenberg
8c5f117882
RELEASE-NOTES: synced 2023-08-02 08:37:42 +02:00
Daniel Stenberg
d3142b5729
resolve: use PF_INET6 family lookups when CURL_IPRESOLVE_V6 is set
Previously it would always do PF_UNSPEC if CURL_IPRESOLVE_V4 is not
used, thus unnecessarily asking for addresses that will not be used.

Reported-by: Joseph Tharayil
Fixes #11564
Closes #11565
2023-08-02 00:49:16 +02:00
Daniel Stenberg
fc1e42190f
docs: link to the website versions instead of markdowns
... to make the links work when the markdown is converted to webpages on
https://curl.se

Reported-by: Maurício Meneghini Fauth
Fixes https://github.com/curl/curl-www/issues/272
Closes #11569
2023-08-02 00:47:41 +02:00
Viktor Szakats
00f8f9c22b
cmake: cache more config and delete unused ones
- cache more Windows config results for faster initialization.

- delete unused config macros `HAVE_SYS_UTSNAME_H`, `HAVE_SSL_H`.

- delete dead references to `sys/utsname.h`.

Closes #11551
2023-08-01 21:59:00 +00:00
Viktor Szakats
c09466abff
egd: delete feature detection and related source code
EGD is Entropy Gathering Daemon, a socket-based entropy source supported
by pre-OpenSSL v1.1 versions and now deprecated. curl also deprecated it
a while ago.

Its detection in CMake was broken all along because OpenSSL libs were
not linked at the point of feature check.

Delete detection from both cmake and autotools, along with the related
source snippet, and the `--with-egd-socket=` `./configure` option.

Closes #11556
2023-08-01 21:58:56 +00:00
Stefan Eissing
bb07240263
tests: fix h3 server check and parallel instances
- fix check for availability of nghttpx server
- add `tcp` frontend config for same port as quic, as
  without this, port 3000 is bound which clashes for parallel
  testing

Closes #11553
2023-08-01 16:11:48 +02:00
Daniel Stenberg
f4ff8072f2
docs/cmdline-opts: spellfixes, typos and polish
To make them accepted by the spell checker

Closes #11562
2023-08-01 14:12:22 +02:00
Daniel Stenberg
6b11d9d86b
CI/spellcheck: build curl.1 and spellcheck it
Added acceptable words

Closes #11562
2023-08-01 14:12:03 +02:00
Alexander Jaeger
959e613e7c
misc: fix various typos
Closes #11561
2023-08-01 14:09:32 +02:00
Daniel Stenberg
d4618a3fc4
http2: avoid too early connection re-use/multiplexing
HTTP/1 connections that are upgraded to HTTP/2 should not be picked up
for reuse and multiplexing by other handles until the 101 switching
process is completed.

Lots-of-debgging-by: Stefan Eissing
Reported-by: Richard W.M. Jones
Bug: https://curl.se/mail/lib-2023-07/0045.html
Closes #11557
2023-08-01 11:30:07 +02:00
Daniel Stenberg
15c40a32b7
Revert "KNOWN_BUGS: build for iOS simulator on macOS 13.2 with Xcode 14"
This reverts commit 2e8a3d7cb7.

It's a user error for supplying incomplete information to the build system.

Reported-by: Ryan Schmidt
Ref: https://github.com/curl/curl/issues/11215#issuecomment-1658729367
2023-08-01 10:22:39 +02:00
Viktor Szakats
2ebc74c36a
cmake: add support for single libcurl compilation pass
Before this patch CMake builds used two separate compilation passes to
build the shared and static libcurl respectively. This patch allows to
reduce that to a single pass if the target platform and build settings
allow it.

This reduces CMake build times when building both static and shared
libcurl at the same time, making these dual builds an almost zero-cost
option.

Enable this feature for Windows builds, where the difference between the
two passes was the use of `__declspec(dllexport)` attribute for exported
API functions for the shared builds. This patch replaces this method
with the use of `libcurl.def` at DLL link time.

Also update `Makefile.mk` to use `libcurl.def` to export libcurl API
symbols on Windows. This simplifies (or fixes) this build method (e.g.
in curl-for-win, which generated a `libcurl.def` from `.h` files using
an elaborate set of transformations).

`libcurl.def` has the maintenance cost of keeping the list of public
libcurl API symbols up-to-date. This list seldom changes, so the cost
is low.

Closes #11546
2023-08-01 08:12:26 +00:00
Viktor Szakats
dee310d542
cmake: detect SSL_set0_wbio in OpenSSL
Present in OpenSSL 1.1.0 and BoringSSL.
Missing from LibreSSL 3.8.0.

Follow-up to f39472ea9f

While here, also fix `RAND_egd()` detection which was broken, likely all
along. This feature is probably broken with CMake builds and also
requires a sufficiently obsolete OpenSSL version, so this part of the
update was not tested.

Closes #11555
2023-08-01 08:12:18 +00:00
Viktor Szakats
4a67b23f33
cmake: fixup H2 duplicate symbols for unity builds
Closes #11550
2023-08-01 08:12:07 +00:00
Pablo Busse
11d46bcfb2 openssl: Support async cert verify callback
- Update the OpenSSL connect state machine to handle
  SSL_ERROR_WANT_RETRY_VERIFY.

This allows libcurl users that are using custom certificate validation
to suspend processing while waiting for external I/O during certificate
validation.

Closes https://github.com/curl/curl/pull/11499
2023-08-01 03:49:07 -04:00
Jay Satiro
af3f4e419b tool_cb_wrt: fix invalid unicode for windows console
- Suppress an incomplete UTF-8 sequence at the end of the buffer.

- Attempt to reconstruct incomplete UTF-8 sequence from prior call(s)
  in current call.

Prior to this change, in Windows console UTF-8 sequences split between
two or more calls to the write callback would cause invalid "replacement
characters" U+FFFD to be printed instead of the actual Unicode
character. This is because in Windows only UTF-16 encoded characters are
printed to the console, therefore we convert the UTF-8 contents to
UTF-16, which cannot be done with partial UTF-8 sequences.

Reported-by: Maksim Arhipov

Fixes https://github.com/curl/curl/issues/9841
Closes https://github.com/curl/curl/pull/10890
2023-08-01 03:28:12 -04:00
Daniel Stenberg
0b947e8ca2
sectransp: prevent CFRelease() of NULL
When SecCertificateCopyCommonName() returns NULL, the common_name
pointer remains set to NULL which apparently when calling CFRelease() on
(sometimes?) crashes.

Reported-by: Guillaume Algis
Fixes #9194
Closes #11554
2023-08-01 08:17:26 +02:00
Jay Satiro
2fcb674cdf
vtls: clarify "ALPN: offers" message
Before:
* ALPN: offers h2,http/1.1

After:
* ALPN: curl offers h2,http/1.1

Bug: https://curl.se/mail/lib-2023-07/0041.html
Reported-by: Richard W.M. Jones
Closes #11544
2023-08-01 08:16:14 +02:00
Daniel Stenberg
49e2443186
urlapi: make sure zoneid is also duplicated in curl_url_dup
Add several curl_url_dup() tests to the general lib1560 test.

Reported-by: Rutger Broekhoff
Bug: https://curl.se/mail/lib-2023-07/0047.html
Closes #11549
2023-08-01 08:00:28 +02:00
Sergey
a21f318992
urlapi: fix heap buffer overflow
`u->path = Curl_memdup(path, pathlen + 1);` accesses bytes after the null-terminator.

```
==2676==ERROR: AddressSanitizer: heap-buffer-overflow on address 0x04d48c75 at pc 0x0112708a bp 0x006fb7e0 sp 0x006fb3c4
READ of size 78 at 0x04d48c75 thread T0
    #0 0x1127089 in __asan_wrap_memcpy D:\a\_work\1\s\src\vctools\asan\llvm\compiler-rt\lib\sanitizer_common\sanitizer_common_interceptors.inc:840
    #1 0x1891a0e in Curl_memdup C:\actions-runner\_work\client\client\third_party\curl\lib\strdup.c:97
    #2 0x18db4b0 in parseurl C:\actions-runner\_work\client\client\third_party\curl\lib\urlapi.c:1297
    #3 0x18db819 in parseurl_and_replace C:\actions-runner\_work\client\client\third_party\curl\lib\urlapi.c:1342
    #4 0x18d6e39 in curl_url_set C:\actions-runner\_work\client\client\third_party\curl\lib\urlapi.c:1790
    #5 0x1877d3e in parseurlandfillconn C:\actions-runner\_work\client\client\third_party\curl\lib\url.c:1768
    #6 0x1871acf in create_conn C:\actions-runner\_work\client\client\third_party\curl\lib\url.c:3403
    #7 0x186d8dc in Curl_connect C:\actions-runner\_work\client\client\third_party\curl\lib\url.c:3888
    #8 0x1856b78 in multi_runsingle C:\actions-runner\_work\client\client\third_party\curl\lib\multi.c:1982
    #9 0x18531e3 in curl_multi_perform C:\actions-runner\_work\client\client\third_party\curl\lib\multi.c:2756
```

Closes #11560
2023-08-01 07:59:07 +02:00
Daniel Stenberg
1032f56efa
curl: make %output{} in -w specify a file to write to
It can be used multiple times. Use %output{>>name} to append.

Add docs. Test 990 and 991 verify.

Idea: #11400
Suggested-by: ed0d2b2ce19451f2
Closes #11416
2023-07-31 23:10:59 +02:00
Daniel Stenberg
92ac5a8d59
RELEASE-NOTES: synced 2023-07-31 11:58:26 +02:00
Daniel Stenberg
2e160c9c65
tool: add "variable" support
Add support for command line variables. Set variables with --variable
name=content or --variable name@file (where "file" can be stdin if set
to a single dash (-)).

Variable content is expanded in option parameters using "{{name}}"
(without the quotes) if the option name is prefixed with
"--expand-". This gets the contents of the variable "name" inserted, or
a blank if the name does not exist as a variable. Insert "{{" verbatim
in the string by prefixing it with a backslash, like "\\{{".

Import an environment variable with --variable %name. It makes curl exit
with an error if the environment variable is not set. It can also rather
get a default value if the variable does not exist, using =content or
@file like shown above.

Example: get the USER environment variable into the URL:

 --variable %USER
 --expand-url = "https://example.com/api/{{USER}}/method"

When expanding variables, curl supports a set of functions that can make
the variable contents more convenient to use. It can trim leading and
trailing white space with "trim", output the contents as a JSON quoted
string with "json", URL encode it with "url" and base 64 encode it with
"b64". To apply functions to a variable expansion, add them colon
separated to the right side of the variable. They are then performed in
a left to right order.

Example: get the contents of a file called $HOME/.secret into a variable
called "fix". Make sure that the content is trimmed and percent-encoded
sent as POST data:

  --variable %HOME=/home/default
  --expand-variable fix@{{HOME}}/.secret
  --expand-data "{{fix:trim:url}}"
  https://example.com/

Documented. Many new test cases.

Co-brainstormed-by: Emanuele Torre
Assisted-by: Jat Satiro
Closes #11346
2023-07-31 11:51:34 +02:00
Daniel Stenberg
47a3e6e577
KNOWN_BUGS: cygwin: make install installs curl-config.1 twice
Closes #8839
2023-07-31 08:55:26 +02:00
Daniel Stenberg
2e8a3d7cb7
KNOWN_BUGS: build for iOS simulator on macOS 13.2 with Xcode 14
Closes #11215
2023-07-31 08:53:55 +02:00
Daniel Stenberg
75afa92dc7
KNOWN_BUGS: cmake outputs: no version information available
Closes #11158
2023-07-31 08:51:32 +02:00
Daniel Stenberg
403e4dc1ed
KNOWN_BUGS: APOP authentication fails on POP3
Closes #10073
2023-07-31 08:47:22 +02:00
Daniel Stenberg
d54d5bfcbf
KNOWN_BUGS: hyper is slow
Closes #11203
2023-07-31 08:45:33 +02:00
Patrick Monnerat
038c46f61f
configure, cmake, lib: more form api deprecation
Introduce a --enable-form-api configure option to control its inclusion
in builds. The condition name defined for it is CURL_DISABLE_FORM_API.

Form api code is dependent of MIME: configure and CMake handle this
dependency automatically: CMake by making it a dependent option
explicitly, configure by inheriting the MIME value by default and
rejecting explicit incompatible values.

"form-api" is now a new hidden test feature.

Update libcurl modules to respect this option and adjust tests
accordingly.

Closes #9621
2023-07-31 08:31:38 +02:00
Daniel Stenberg
50e563253d
mailmap: add Derzsi Dániel 2023-07-31 08:29:16 +02:00
Disyer
4f9c20d49f
wolfssl: support loading system CA certificates
Closes #11452
2023-07-31 08:27:50 +02:00
Viktor Szakats
86eff0b0d5
nss: delete more NSS references
Fix the distcheck CI failure and delete more NSS references.

Follow-up to 7c8bae0d9c

Reviewed-by: Marcel Raad
Reviewed-by: Daniel Stenberg
Closes #11548
2023-07-30 21:55:29 +00:00
Daniel Stenberg
7c8bae0d9c
nss: remove support for this TLS library
Closes #11459
2023-07-29 23:44:28 +02:00
Ryan Schmidt
8b7cbe9dec
macOS: fix target detection more
Now SCDynamicStoreCopyProxies is called (and the required frameworks are
linked in) on all versions of macOS and only on macOS. Fixes crash due
to undefined symbol when built with the macOS 10.11 SDK or earlier.

CURL_OSX_CALL_COPYPROXIES is renamed to CURL_MACOS_CALL_COPYPROXIES and
is now only defined when SCDynamicStoreCopyProxies will actually be
called. Previously, it was defined when ENABLE_IPV6 was not defined but
SCDynamicStoreCopyProxies is not called in that case.

TARGET_OS_OSX is only defined in the macOS 10.12 SDK and later and only
when dynamic targets are enabled. TARGET_OS_MAC is always defined but
means any Mac OS or derivative including macOS, iOS, tvOS, and watchOS.
TARGET_OS_IPHONE means any Darwin OS other than macOS.

Follow-up to c73b2f82

Fixes #11502
Closes #11516
2023-07-29 23:42:53 +02:00
Daniel Stenberg
29bce9857a
tool_operate: allow SSL_CERT_FILE and SSL_CERT_DIR
... used at once.

Reported-by: Gabriel Corona
Fixes #11325
Closes #11531
2023-07-29 23:41:42 +02:00
Thomas M. DuBuisson
42697e377e
CI: remove Lift's configuration
The Lift tool is being retired. Their site reads:

"Sonatype Lift will be retiring on Sep 12, 2023, with its analysis
stopping on Aug 12, 2023."

Closes #11541
2023-07-29 23:38:04 +02:00
Nathan Moinvaziri
f540a39b8b Revert "schannel: reverse the order of certinfo insertions"
This reverts commit 8986df802d.

Windows does not guarantee a particular certificate ordering, even
though TLS may have its own ordering/relationship guarantees. Recent
versions of Windows 11 reversed the ordering of ceritifcates returned by
CertEnumCertificatesInStore, therefore this commit no longer works as
initially intended. libcurl makes no guarantees about certificate
ordering if the operating system can't.

Ref: https://github.com/curl/curl/issues/9706

Closes https://github.com/curl/curl/pull/11536
2023-07-29 16:06:16 -04:00
wangzhikun
7bfb10c180 winbuild: improve check for static zlib
- Check for zlib static library name zlibstatic.lib.

zlib's static library has a different name depending on how it was
built. zlibstatic.lib is output by cmake. zlibstat.lib is output by
their pre-generated Visual Studio project files (in the contrib
directory) and defines ZLIB_WINAPI (ie it's meant to use stdcall
instead of cdecl if you end up exporting the zlib functions).

Prior to this change the makefile only checked for the latter.

Closes https://github.com/curl/curl/pull/11521
2023-07-29 15:05:10 -04:00