Commit Graph

30684 Commits

Author SHA1 Message Date
Pablo Busse
11d46bcfb2 openssl: Support async cert verify callback
- Update the OpenSSL connect state machine to handle
  SSL_ERROR_WANT_RETRY_VERIFY.

This allows libcurl users that are using custom certificate validation
to suspend processing while waiting for external I/O during certificate
validation.

Closes https://github.com/curl/curl/pull/11499
2023-08-01 03:49:07 -04:00
Jay Satiro
af3f4e419b tool_cb_wrt: fix invalid unicode for windows console
- Suppress an incomplete UTF-8 sequence at the end of the buffer.

- Attempt to reconstruct incomplete UTF-8 sequence from prior call(s)
  in current call.

Prior to this change, in Windows console UTF-8 sequences split between
two or more calls to the write callback would cause invalid "replacement
characters" U+FFFD to be printed instead of the actual Unicode
character. This is because in Windows only UTF-16 encoded characters are
printed to the console, therefore we convert the UTF-8 contents to
UTF-16, which cannot be done with partial UTF-8 sequences.

Reported-by: Maksim Arhipov

Fixes https://github.com/curl/curl/issues/9841
Closes https://github.com/curl/curl/pull/10890
2023-08-01 03:28:12 -04:00
Daniel Stenberg
0b947e8ca2
sectransp: prevent CFRelease() of NULL
When SecCertificateCopyCommonName() returns NULL, the common_name
pointer remains set to NULL which apparently when calling CFRelease() on
(sometimes?) crashes.

Reported-by: Guillaume Algis
Fixes #9194
Closes #11554
2023-08-01 08:17:26 +02:00
Jay Satiro
2fcb674cdf
vtls: clarify "ALPN: offers" message
Before:
* ALPN: offers h2,http/1.1

After:
* ALPN: curl offers h2,http/1.1

Bug: https://curl.se/mail/lib-2023-07/0041.html
Reported-by: Richard W.M. Jones
Closes #11544
2023-08-01 08:16:14 +02:00
Daniel Stenberg
49e2443186
urlapi: make sure zoneid is also duplicated in curl_url_dup
Add several curl_url_dup() tests to the general lib1560 test.

Reported-by: Rutger Broekhoff
Bug: https://curl.se/mail/lib-2023-07/0047.html
Closes #11549
2023-08-01 08:00:28 +02:00
Sergey
a21f318992
urlapi: fix heap buffer overflow
`u->path = Curl_memdup(path, pathlen + 1);` accesses bytes after the null-terminator.

```
==2676==ERROR: AddressSanitizer: heap-buffer-overflow on address 0x04d48c75 at pc 0x0112708a bp 0x006fb7e0 sp 0x006fb3c4
READ of size 78 at 0x04d48c75 thread T0
    #0 0x1127089 in __asan_wrap_memcpy D:\a\_work\1\s\src\vctools\asan\llvm\compiler-rt\lib\sanitizer_common\sanitizer_common_interceptors.inc:840
    #1 0x1891a0e in Curl_memdup C:\actions-runner\_work\client\client\third_party\curl\lib\strdup.c:97
    #2 0x18db4b0 in parseurl C:\actions-runner\_work\client\client\third_party\curl\lib\urlapi.c:1297
    #3 0x18db819 in parseurl_and_replace C:\actions-runner\_work\client\client\third_party\curl\lib\urlapi.c:1342
    #4 0x18d6e39 in curl_url_set C:\actions-runner\_work\client\client\third_party\curl\lib\urlapi.c:1790
    #5 0x1877d3e in parseurlandfillconn C:\actions-runner\_work\client\client\third_party\curl\lib\url.c:1768
    #6 0x1871acf in create_conn C:\actions-runner\_work\client\client\third_party\curl\lib\url.c:3403
    #7 0x186d8dc in Curl_connect C:\actions-runner\_work\client\client\third_party\curl\lib\url.c:3888
    #8 0x1856b78 in multi_runsingle C:\actions-runner\_work\client\client\third_party\curl\lib\multi.c:1982
    #9 0x18531e3 in curl_multi_perform C:\actions-runner\_work\client\client\third_party\curl\lib\multi.c:2756
```

Closes #11560
2023-08-01 07:59:07 +02:00
Daniel Stenberg
1032f56efa
curl: make %output{} in -w specify a file to write to
It can be used multiple times. Use %output{>>name} to append.

Add docs. Test 990 and 991 verify.

Idea: #11400
Suggested-by: ed0d2b2ce19451f2
Closes #11416
2023-07-31 23:10:59 +02:00
Daniel Stenberg
92ac5a8d59
RELEASE-NOTES: synced 2023-07-31 11:58:26 +02:00
Daniel Stenberg
2e160c9c65
tool: add "variable" support
Add support for command line variables. Set variables with --variable
name=content or --variable name@file (where "file" can be stdin if set
to a single dash (-)).

Variable content is expanded in option parameters using "{{name}}"
(without the quotes) if the option name is prefixed with
"--expand-". This gets the contents of the variable "name" inserted, or
a blank if the name does not exist as a variable. Insert "{{" verbatim
in the string by prefixing it with a backslash, like "\\{{".

Import an environment variable with --variable %name. It makes curl exit
with an error if the environment variable is not set. It can also rather
get a default value if the variable does not exist, using =content or
@file like shown above.

Example: get the USER environment variable into the URL:

 --variable %USER
 --expand-url = "https://example.com/api/{{USER}}/method"

When expanding variables, curl supports a set of functions that can make
the variable contents more convenient to use. It can trim leading and
trailing white space with "trim", output the contents as a JSON quoted
string with "json", URL encode it with "url" and base 64 encode it with
"b64". To apply functions to a variable expansion, add them colon
separated to the right side of the variable. They are then performed in
a left to right order.

Example: get the contents of a file called $HOME/.secret into a variable
called "fix". Make sure that the content is trimmed and percent-encoded
sent as POST data:

  --variable %HOME=/home/default
  --expand-variable fix@{{HOME}}/.secret
  --expand-data "{{fix:trim:url}}"
  https://example.com/

Documented. Many new test cases.

Co-brainstormed-by: Emanuele Torre
Assisted-by: Jat Satiro
Closes #11346
2023-07-31 11:51:34 +02:00
Daniel Stenberg
47a3e6e577
KNOWN_BUGS: cygwin: make install installs curl-config.1 twice
Closes #8839
2023-07-31 08:55:26 +02:00
Daniel Stenberg
2e8a3d7cb7
KNOWN_BUGS: build for iOS simulator on macOS 13.2 with Xcode 14
Closes #11215
2023-07-31 08:53:55 +02:00
Daniel Stenberg
75afa92dc7
KNOWN_BUGS: cmake outputs: no version information available
Closes #11158
2023-07-31 08:51:32 +02:00
Daniel Stenberg
403e4dc1ed
KNOWN_BUGS: APOP authentication fails on POP3
Closes #10073
2023-07-31 08:47:22 +02:00
Daniel Stenberg
d54d5bfcbf
KNOWN_BUGS: hyper is slow
Closes #11203
2023-07-31 08:45:33 +02:00
Patrick Monnerat
038c46f61f
configure, cmake, lib: more form api deprecation
Introduce a --enable-form-api configure option to control its inclusion
in builds. The condition name defined for it is CURL_DISABLE_FORM_API.

Form api code is dependent of MIME: configure and CMake handle this
dependency automatically: CMake by making it a dependent option
explicitly, configure by inheriting the MIME value by default and
rejecting explicit incompatible values.

"form-api" is now a new hidden test feature.

Update libcurl modules to respect this option and adjust tests
accordingly.

Closes #9621
2023-07-31 08:31:38 +02:00
Daniel Stenberg
50e563253d
mailmap: add Derzsi Dániel 2023-07-31 08:29:16 +02:00
Disyer
4f9c20d49f
wolfssl: support loading system CA certificates
Closes #11452
2023-07-31 08:27:50 +02:00
Viktor Szakats
86eff0b0d5
nss: delete more NSS references
Fix the distcheck CI failure and delete more NSS references.

Follow-up to 7c8bae0d9c

Reviewed-by: Marcel Raad
Reviewed-by: Daniel Stenberg
Closes #11548
2023-07-30 21:55:29 +00:00
Daniel Stenberg
7c8bae0d9c
nss: remove support for this TLS library
Closes #11459
2023-07-29 23:44:28 +02:00
Ryan Schmidt
8b7cbe9dec
macOS: fix target detection more
Now SCDynamicStoreCopyProxies is called (and the required frameworks are
linked in) on all versions of macOS and only on macOS. Fixes crash due
to undefined symbol when built with the macOS 10.11 SDK or earlier.

CURL_OSX_CALL_COPYPROXIES is renamed to CURL_MACOS_CALL_COPYPROXIES and
is now only defined when SCDynamicStoreCopyProxies will actually be
called. Previously, it was defined when ENABLE_IPV6 was not defined but
SCDynamicStoreCopyProxies is not called in that case.

TARGET_OS_OSX is only defined in the macOS 10.12 SDK and later and only
when dynamic targets are enabled. TARGET_OS_MAC is always defined but
means any Mac OS or derivative including macOS, iOS, tvOS, and watchOS.
TARGET_OS_IPHONE means any Darwin OS other than macOS.

Follow-up to c73b2f82

Fixes #11502
Closes #11516
2023-07-29 23:42:53 +02:00
Daniel Stenberg
29bce9857a
tool_operate: allow SSL_CERT_FILE and SSL_CERT_DIR
... used at once.

Reported-by: Gabriel Corona
Fixes #11325
Closes #11531
2023-07-29 23:41:42 +02:00
Thomas M. DuBuisson
42697e377e
CI: remove Lift's configuration
The Lift tool is being retired. Their site reads:

"Sonatype Lift will be retiring on Sep 12, 2023, with its analysis
stopping on Aug 12, 2023."

Closes #11541
2023-07-29 23:38:04 +02:00
Nathan Moinvaziri
f540a39b8b Revert "schannel: reverse the order of certinfo insertions"
This reverts commit 8986df802d.

Windows does not guarantee a particular certificate ordering, even
though TLS may have its own ordering/relationship guarantees. Recent
versions of Windows 11 reversed the ordering of ceritifcates returned by
CertEnumCertificatesInStore, therefore this commit no longer works as
initially intended. libcurl makes no guarantees about certificate
ordering if the operating system can't.

Ref: https://github.com/curl/curl/issues/9706

Closes https://github.com/curl/curl/pull/11536
2023-07-29 16:06:16 -04:00
wangzhikun
7bfb10c180 winbuild: improve check for static zlib
- Check for zlib static library name zlibstatic.lib.

zlib's static library has a different name depending on how it was
built. zlibstatic.lib is output by cmake. zlibstat.lib is output by
their pre-generated Visual Studio project files (in the contrib
directory) and defines ZLIB_WINAPI (ie it's meant to use stdcall
instead of cdecl if you end up exporting the zlib functions).

Prior to this change the makefile only checked for the latter.

Closes https://github.com/curl/curl/pull/11521
2023-07-29 15:05:10 -04:00
Daniel Stenberg
756c6a02ba
configure: use the pkg-config --libs-only-l flag for libssh2
... instead of --libs, as that one also returns -L flags.

Reported-by: Wilhelm von Thiele
Fixes #11538
Closes #11539
2023-07-29 12:52:24 +02:00
Viktor Szakats
1199308dbc
cmake: support building static and shared libcurl in one go
This patch adds the ability to build a static and shared libcurl library
in a single build session. It also adds an option to select which one to
use when building the curl executable.

New build options:
- `BUILD_STATIC_LIBS`. Default: `OFF`.
  Enabled automatically if `BUILD_SHARED_LIBS` is `OFF`.
- `BUILD_STATIC_CURL`. Default: `OFF`.
  Requires `BUILD_STATIC_LIBS` enabled.
  Enabled automatically if building static libcurl only.
- `STATIC_LIB_SUFFIX`. Default: empty.
- `IMPORT_LIB_SUFFIX`. Default: `_imp` if implib filename would collide
  with static lib name (typically with MSVC) in Windows builds.
  Otherwise empty.

Also:

- Stop setting the `CURL_STATICLIB` macro via `curl_config.h`, and pass
  it directly to the compiler. This also allows to delete a condition
  from `tests/server/CMakeLists.txt`.

- Complete a TODO by following the logic used in autotools (also for
  `LIBCURL_NO_SHARED`), and set `-DCURL_STATICLIB` in `Cflags:` of
  `libcurl.pc` for _static-only_ curl builds.

- Convert an existing CI test to build both shared and static libcurl.

Closes #11505
2023-07-29 00:40:01 +00:00
Stefan Eissing
40c63290e3
CI/awslc: add cache for build awslc library
Closes #11535
2023-07-28 23:13:13 +02:00
Stefan Eissing
b0f3e030c0
GHA/linux.yml: add caching
Closes #11532
2023-07-28 23:10:41 +02:00
Daniel Stenberg
d6d5c46769
RELEASE-NOTES: synced
Bump working version to 8.3.0
2023-07-27 23:58:57 +02:00
Daniel Stenberg
5413d24aae
url: remove infof() output for "still name resolving"
The message does not help and might get spewed a lot during times.

Reported-by: yushicheng7788 on github
Fixes #11394
Closes #11529
2023-07-27 23:55:21 +02:00
Daniel Stenberg
775018d273
KNOWN_BUGS: cygwin: "WARNING: UNPROTECTED PRIVATE KEY FILE!"
Closes #11244
2023-07-27 23:43:28 +02:00
Stefan Eissing
2284104c3f
CI: quiche updates
- remove quiche from standard `linux` workflow
- add mod_h2 caching to quiche workflow
- rename quiche to quiche-linux
- move version definitions into env section

Closes #11528
2023-07-27 17:47:09 +02:00
Stefan Eissing
63936d9ff8
http2: disable asssertion blocking OSSFuzz testing
- not clear how this triggers and it blocks OSSFuzz testing other
  things. Since we handle the case with an error return, disabling the
  assertion for now seems the best way forward.

Fixes #11500
Closes #11519
2023-07-27 17:44:15 +02:00
Stefan Eissing
a05cce74f9
http2: fix in h2 proxy tunnel: progress in ingress on sending
- depending on what is tunneled, the proxy may never get invoked for
  receiving data explicitly. Not progressing ingress may lead to stalls
  due to missed WINDOW_UPDATEs.

CI:
- add a chache for building mod_h2

Closes #11527
2023-07-27 17:00:38 +02:00
Stefan Eissing
3f518ce4b1
CI ngtcp2+quictls: use nghttpx cache as in quiche build 2023-07-27 16:59:37 +02:00
Jay Satiro
691c32f6f5 bearssl: don't load CA certs when peer verification is disabled
We already do this for other SSL backends.

Bug: https://github.com/curl/curl/pull/11457#issuecomment-1644587473
Reported-by: kyled-dell@users.noreply.github.com

Closes https://github.com/curl/curl/pull/11497
2023-07-27 03:39:19 -04:00
Daniel Stenberg
392b73ac99
easy: remove #ifdefs to make code easier on the eye
Closes #11525
2023-07-26 23:57:08 +02:00
Stefan Eissing
bea86dad6e
GHA: adding quiche workflow
- adding separate quiche workflow to also build nghttpx server for testing

Closes #11517
2023-07-26 18:19:30 +02:00
Daniel Stenberg
50490c0679
RELEASE-NOTES: synced
curl 8.2.1 release
2023-07-26 08:10:40 +02:00
Daniel Stenberg
a01f89d7d9
THANKS: add contributors from 8.2.1 2023-07-26 08:10:40 +02:00
Daniel Stenberg
934da11c0d
docs: provide more see also for cipher options
More cross references. Hide nroff errors.

Closes #11513
2023-07-25 22:30:09 +02:00
Daniel Stenberg
7b20ba311d
docs: mark two TLS options for TLS, not SSL
Closes #11514
2023-07-25 17:22:02 +02:00
bch
c06d7e25bf
curl_multi_wait.3: fix arg quoting to doc macro .BR
Closes #11511
2023-07-25 10:29:39 +02:00
Daniel Stenberg
ead8df50cd
RELEASE-NOTES: synced 2023-07-24 23:03:23 +02:00
Viktor Szakats
a15ef19a8c
cmake: update ngtcp2 detection
Replace `OpenSSL` with `quictls` to follow the same change
in the v0.17.0 ngtcp2 release.

Follow-up to e0093b4b73

Closes #11508
2023-07-24 18:13:53 +00:00
Stefan Eissing
c76df46a19
http: VLH, very large header test and fixes
- adding tests using very large passwords in auth
- fixes general http sending to treat h3 like h2, and
  not like http1.1
- eliminate H2_HEADER max definitions and use the commmon
  DYN_HTTP_REQUEST everywhere, different limits do not help
- fix http2 handling of requests denied by nghttp2 on send
  to immediately report the refused stream

Closes #11509
2023-07-24 19:25:20 +02:00
Andrei Rybak
3c0a91077c
CONTRIBUTE: drop mention of copyright year ranges
Year ranges in copyrights were dropped in commits [1] and [2].
Verification of year ranges in copyrights was dropped from script
'scripts/copyright.pl' in commit [3].  However, the corresponding
passages in file 'docs/CONTRIBUTE.md' weren't updated.

Drop mentions of copyright year ranges from 'docs/CONTRIBUTE.md'.

[1] 2bc1d775f (copyright: update all copyright lines and remove year
    ranges, 2023-01-02)
[2] c46761bd8 (tests/http: remove year ranges from copyrights,
    2023-03-14)
[3] 0e293bacb (copyright.pl: cease doing year verifications, 2023-01-28)

Closes #11504
2023-07-23 23:47:35 +02:00
Andrei Rybak
971f03e5c0
CONTRIBUTE: fix syntax in commit message description
File 'docs/CONTRIBUTE.md' includes a description of how one should write
commit messages in the curl project.  Different possible parts of the
message are enclosed in square brackets.  One exception is the section
describing how the curl project doesn't use "Signed-off-by" commit
trailers [1], which is enclosed in an opening curly brace paired with a
closing square bracket.

Fix the enclosing square brackets in description of "Signed-off-by"
trailers in commit messages in file 'docs/CONTRIBUTE.md'.

[1] See description of option '--signoff' in Git documentation:
    https://git-scm.com/docs/git-commit

Closes #11504
2023-07-23 23:47:16 +02:00
Daniel Stenberg
e310497a9a
src/mkhelp: strip off escape sequences
At some point the nroff command stopped stripping off escape sequences,
so then this script needs to do the job instead.

Reported-by: VictorVG on github
Fixes #11501
Closes #11503
2023-07-23 23:30:51 +02:00
Daniel Stenberg
57f56cc374
KNOWN_BUGS: building for old macOS fails with gcc
Closes #11441
2023-07-23 17:00:29 +02:00