Commit Graph

31624 Commits

Author SHA1 Message Date
Daniel Stenberg
30e04beafe
RELEASE-NOTES: synced 2023-11-02 16:30:15 +01:00
Daniel Stenberg
63e1a9a1e7
docs: clarify that curl passes on input unfiltered
... for several options.

Reported-by: Ophir Lojkine

Closes #12249
2023-11-02 16:27:39 +01:00
Daniel Stenberg
5c846a12a3
urlapi: when URL encoding the fragment, pass in the right length
A benign bug because it would only add an extra null terminator.

Made lib1560 get a test that runs this code.

Closes #12250
2023-11-02 16:23:17 +01:00
Stefan Eissing
bbdbd4b025
vtls: late clone of connection ssl config
- perform connection cache matching against `data->set.ssl.primary`
  and proxy counterpart
- fully clone connection ssl config only when connection is used

Closes #12237
2023-11-02 14:30:42 +01:00
Stefan Eissing
5d0b3eda43
msh3: error when built with CURL_DISABLE_SOCKETPAIR set
Reported-by: Gisle Vanem
Closes #12252
Fixes #12213
2023-11-02 14:29:21 +01:00
Daniel Stenberg
8c058a820a
hsts: skip single-dot hostname
Reported-by: Maksymilian Arciemowicz

Closes #12247
2023-11-02 13:17:45 +01:00
Daniel Stenberg
ddb95c155e
vtls: fix build without proxy
Follow-up to bf0e278a3c

closes #12243
2023-11-01 14:55:04 +01:00
Daniel Stenberg
41b9fd5384
docs/example/keepalive.c: show TCP keep-alive options
Closes #12242
2023-11-01 10:57:15 +01:00
Daniel Stenberg
8c8a03f252
lib1560: verify appending blank URL encoded query string 2023-11-01 10:55:58 +01:00
Daniel Stenberg
ffbc9981c4
urlapi: skip appending NULL pointer query
Reported-by: kirbyn17 on hackerone

Closes #12240
2023-11-01 10:55:55 +01:00
Daniel Stenberg
21c5d5971e
lib1560: verify setting host to "" with and without URL encode 2023-11-01 10:55:55 +01:00
Daniel Stenberg
c64d0d67fd
urlapi: avoid null deref if setting blank host to url encode
Reported-by: kirbyn17 on hackerone

Closes #12240
2023-11-01 10:55:46 +01:00
Daniel Stenberg
27f2352d49
dynbuf: assert for NULL pointer inputs
Help us catch more mistakes.

Closes #12238
2023-11-01 10:53:49 +01:00
Daniel Stenberg
5f78cf503c
HTTP3: ngtcp2 builds are no longer experimental
The other HTTP/3 backends are still experimental.

Closes #12235
2023-10-31 14:31:46 +01:00
Stefan Eissing
bf0e278a3c
vtls: cleanup SSL config management
- remove `Curl_ssl_get_config()`, no longer needed

Closes #12204
2023-10-31 14:29:36 +01:00
Daniel Stenberg
faa45a637f
libcurl-thread.3: simplify the TLS section
All TLS libraries curl can use are threadsafe since OpenSSL 1.1.x, August
2016.

Closes #12233
2023-10-31 14:27:25 +01:00
Daniel Stenberg
d2d48f21f3
configure: better --disable-http
- disable HTTPS-proxy as well, since it can't work without HTTP

- curl_setup: when HTTP is disabled, also disable all features that are
  HTTP-only

- version: HTTPS-proxy only exists if HTTP support exists

Closes #12223
2023-10-30 17:02:13 +01:00
Daniel Stenberg
225db9196a
http: consider resume with CURLOPT_FAILONERRROR and 416 to be fine
Finding a 'Content-Range:' in the response changed the handling.

Add test case 1475 to verify -C - with 416 and Content-Range: header,
which is almost exactly like test 194 which instead uses a fixed -C
offset. Adjusted test 194 to also be considered fine.

Fixes #10521
Reported-by: Smackd0wn
Fixes #12174
Reported-by: Anubhav Rai
Closes #12176
2023-10-30 17:00:34 +01:00
Stefan Eissing
5a3e277290
GHA: fix checkout of quictls repository to use correct branch name
Follow-up to c868b0e30f

Closes #12232
2023-10-30 16:58:45 +01:00
Daniel Stenberg
92a315b1a9
docs/example/localport.c: show off CURLOPT_LOCALPORT
Closes #12230
2023-10-30 12:49:00 +01:00
Daniel Stenberg
22130826b7
docs/examples/interface.c: show CURLOPT_INTERFACE use
Although super simple.

Closes #12229
2023-10-30 12:45:34 +01:00
Viktor Szakats
aca7d808c3
build: fix compiler warning with auths disabled
```
./curl/lib/http.c:979:12: warning: unused function 'is_valid_auth_separator' [-Wunused-function]
static int is_valid_auth_separator(char ch)
           ^
5 warnings generated.
```

Follow-up to e92edfbef6 #11490

Closes #12227
2023-10-30 11:00:19 +00:00
Viktor Szakats
960d601481
build: require Windows XP or newer
After this patch we assume availability of `getaddrinfo` and
`freeaddrinfo`, first introduced in Windows XP. Meaning curl
now requires building for Windows XP as a minimum.

TODO: assume these also in autotools.

Ref: https://github.com/curl/curl/pull/12221#issuecomment-1783761806
Closes #12225
2023-10-30 10:46:40 +00:00
Viktor Szakats
4f591db44c
appveyor: bump one job to OpenSSL 3.1 (was 1.1.1)
Use 3.1 with the modern runner image.

We still use 1.1.1 in 8 jobs.

1.1.1 is EOL since 2023-09-11:
https://www.openssl.org/blog/blog/2023/03/28/1.1.1-EOL/

Also:
- add missing SSL-backend to job descriptions.
- tidy up CPU in job descriptions.

Closes #12226
2023-10-30 10:46:40 +00:00
Daniel Stenberg
ab885eccd6
RELEASE-NOTES: synced 2023-10-30 08:48:26 +01:00
Daniel Stenberg
c868b0e30f
GHA: bump ngtcp2, nghttp3, nghttp2 and quictls versions
ngtcp2 1.0.1
nghttp3 1.0.0
nghttp2 1.58.0
quictls 3.1.4+quic

also sync HTTP3.md with these changes

Closes #12132
2023-10-29 17:39:23 +01:00
Kareem
a51fff1850
wolfssl: add default case for wolfssl_connect_step1 switch
Closes #12218
2023-10-29 14:52:19 +01:00
Jay Satiro
904ae12238 curl_setup: disallow Windows IPv6 builds missing getaddrinfo
- On Windows if IPv6 is enabled but getaddrinfo is missing then #error
  the build.

curl can be built with IPv6 support (ENABLE_IPV6) but without the
ability to resolve hosts to IPv6 addresses (HAVE_GETADDRINFO). On
Windows this is highly unlikely and should be considered a bad build
configuration.

Such a bad configuration has already given us a bug that was hard to
diagnose. See #12134 and #12136 for discussion.

Ref: https://github.com/curl/curl/issues/12134
Ref: https://github.com/curl/curl/pull/12136

Closes https://github.com/curl/curl/pull/12221
2023-10-29 03:41:33 -04:00
Nico Rieck
2d6333101a openssl: make CURLSSLOPT_NATIVE_CA import Windows intermediate CAs
- If CURLSSLOPT_NATIVE_CA on Windows then import from intermediate CA
  "CA" store after importing from root CA "ROOT" store.

This change allows curl to work in situations where a server does not
send all intermediate certs and they are present in the "CA" store (the
store with intermediate CAs). This is already allowed by the Schannel
backend.

Also this change makes partial chain verification possible for those
certs since we allow partial chain verification by default for OpenSSL
(unless CURLSSLOPT_NO_PARTIALCHAIN). This is not allowed by the Schannel
backend.

Prior to this change CURLSSLOPT_NATIVE_CA only imported "ROOT" certs.

Fixes https://github.com/curl/curl/issues/12155
Closes https://github.com/curl/curl/pull/12185
2023-10-29 03:39:43 -04:00
Viktor Szakats
c0d4fbb1f5
Makefile.mk: fix -rtmp option for non-Windows [ci skip] 2023-10-28 10:06:43 +00:00
Daniel Stenberg
91188c6480
asyn-ares: handle no connection in the addrinfo callback
To avoid crashing.

Follow-up from 56a4db2
Closes #12219
2023-10-28 10:46:06 +02:00
Jay Satiro
68673c3e9e hostip6: fix DEBUG_ADDRINFO builds
- Removed unused and incorrect parameter from dump_addrinfo().

Bug: https://github.com/curl/curl/commit/56a4db2e#commitcomment-131050442
Reported-by: Gisle Vanem

Closes https://github.com/curl/curl/pull/12212
2023-10-28 00:27:20 -04:00
Viktor Szakats
5839b8ae98
Makefile.mk: restore _mingw.h for default _WIN32_WINNT
In 8.4.0 we deleted `_mingw.h` as part of purging old-mingw support.
Turns out `_mingw.h` had the side-effect of setting a default
`_WIN32_WINNT` value expected by `lib/config-win32.h` to enable
`getaddrinfo` support in `Makefile.mk` mingw-w64 builds. This caused
disabling support for this unless specifying the value manually.

Restore this header and update its comment to tell why we continue
to need it.

This triggered a regression in official Windows curl builds starting
with 8.4.0_1. Fixed in 8.4.0_6. (8.5.0 will be using CMake.)

Regression from 38029101e2 #11625

Reported-by: zhengqwe on github
Helped-by: Nico Rieck
Fixes #12134
Fixes #12136
Closes #12217
2023-10-28 00:10:12 +00:00
Viktor Szakats
d14e11d248
hostip: silence compiler warning -Wparentheses-equality
Seen with LLVM 17.

```
hostip.c:1336:22: warning: equality comparison with extraneous parentheses [-Wparentheses-equality]
 1336 |        (a->ai_family == PF_INET)) {
      |         ~~~~~~~~~~~~~^~~~~~~~~~
hostip.c:1336:22: note: remove extraneous parentheses around the comparison to silence this warning
 1336 |        (a->ai_family == PF_INET)) {
      |        ~             ^         ~
hostip.c:1336:22: note: use '=' to turn this equality comparison into an assignment
 1336 |        (a->ai_family == PF_INET)) {
      |                      ^~
      |                      =
1 warning generated.
```

Follow-up to b651aba096 #12145

Reviewed-by: Daniel Stenberg
Closes #12215
2023-10-27 22:30:26 +00:00
Stefan Eissing
277486b6b4
doh: use PIPEWAIT when HTTP/2 is attempted
Closes #12214
2023-10-27 17:13:24 +02:00
Daniel Stenberg
2478cbbff2
setopt: remove outdated cookie comment
Closes #12206
2023-10-27 16:59:40 +02:00
Stefan Eissing
37b5cf4fa0
cfilter: provide call to tell connection to forget a socket
- fixed libssh.c workaround for a socket being closed by
  the library
- eliminate the terrible hack in cf-socket.c to guess when
  this happened and try not closing the socket again.
- fixes race in eyeballing when socket could have failed to
  be closed for a discarded connect attempt

Closes #12207
2023-10-27 16:59:13 +02:00
Stefan Eissing
39547ae64d
url: protocol handler lookup tidy-up
- rename lookup to what it does
- use ARRAYSIZE instead of NULL check for end
- offer alternate lookup for 0-terminated strings

Closes #12216
2023-10-27 16:55:54 +02:00
Viktor Szakats
a426b5050f
build: variadic macro tidy-ups
- delete unused `HAVE_VARIADIC_MACROS_C99/GCC` feature checks.
  (both autotools and CMake.)
- delete duplicate `NULL` check in `Curl_trc_cf_infof()`.
- fix compiler warning in `CURL_DISABLE_VERBOSE_STRINGS` builds.
  ```
  ./lib/cf-socket.c:122:41: warning: unused parameter 'data' [-Wunused-parameter]
  static void nosigpipe(struct Curl_easy *data,
                                          ^
  ```
- fix `#ifdef` comments in `lib/curl_trc.{c,h}`.
- fix indentation in some `infof()` calls.

Follow-up to dac293cfb7 #12167

Cherry-picked from #12105
Closes #12210
2023-10-27 00:37:34 +00:00
Viktor Szakats
191e695fe4
cmake: speed up threads setup for Windows
Win32 threads are always available. We enabled them unconditionally
(with `ENABLE_THREADED_RESOLVER`). CMake built-in thread detection
logic has this condition hard-coded for Windows as well (since at least
2007).

Instead of doing all the work of detecting pthread combinations on
Windows, then discarding those results, skip these efforts and assume
built-in thread support when building for Windows.

This saves 1-3 slow CMake configuration steps.

Reviewed-by: Daniel Stenberg
Closes #12202
2023-10-27 00:37:34 +00:00
Viktor Szakats
c5d506e9bb
cmake: speed up zstd detection
Before this patch we detected the presence of a specific zstd API to
see if we can use the library. zstd published that API in its first
stable release: v1.0.0 (2016-08-31).

Replace that method by detecting the zstd library version instead and
accepting if it's v1.0.0 or newer. Also display this detected version
and display a warning if the zstd found is unfit for curl.

We use the same version detection method as zstd itself, via its public
C header.

This deviates from autotools which keeps using the slow method of
looking for the API by building a test program. The outcome is the same
as long as zstd keeps offering this API.

Ref: 5a0c8e2439 (2016-08-12, committed)
Ref: https://github.com/facebook/zstd/releases/tag/v0.8.1 (2016-08-18, first released)
Ref: https://github.com/facebook/zstd/releases/tag/v1.0.0

Reviewed-by: Daniel Stenberg
Closes #12200
2023-10-27 00:37:34 +00:00
Daniel Stenberg
9ee6da65d8
openssl: fix infof() to avoid compiler warning for %s with null
vtls/openssl.c: In function ‘ossl_connect_step2’:
../lib/curl_trc.h:120:10: error: ‘%s’ directive argument is null [-Werror=format-overflow=]
  120 |          Curl_infof(data, __VA_ARGS__); } while(0)
      |          ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~
vtls/openssl.c:4008:5: note: in expansion of macro ‘infof’
 4008 |     infof(data, "SSL connection using %s / %s / %s / %s",
      |     ^~~~~
vtls/openssl.c:4008:49: note: format string is defined here
 4008 |     infof(data, "SSL connection using %s / %s / %s / %s",
      |                                                 ^~

Follow-up to b6e6d4ff8f
Closes #12196
2023-10-26 17:43:38 +02:00
Stefan Eissing
dac293cfb7
lib: apache style infof and trace macros/functions
- test for a simplified C99 variadic check
- args to infof() in --disable-verbose are no longer disregarded but
  must compile.

Closes #12167
Fixes #12083
Fixes #11880
Fixes #11891
2023-10-26 17:42:54 +02:00
Daniel Stenberg
d1a7da6531
RELEASE-NOTES: synced 2023-10-26 17:33:25 +02:00
Stefan Eissing
56a4db2e4e
urldata: move async resolver state from easy handle to connectdata
- resolving is done for a connection, not for every transfer
- save create/dup/free of a cares channel for each transfer
- check values of setopt calls against a local channel if no
  connection has been attached yet, when needed.

Closes #12198
2023-10-26 17:29:17 +02:00
Daniel Stenberg
910f740ce2
CURLOPT_WRITEFUNCTION.3: clarify what libcurl returns for CURL_WRITEFUNC_ERROR
It returns CURLE_WRITE_ERROR. It was not previously stated clearly.

Reported-by: enWILLYado on github
Fixes #12201
Closes #12203
2023-10-26 17:08:25 +02:00
Viktor Szakats
6ec70a9dd3
autotools: update references to deleted crypt-auth option
Delete leftovers of the `crypt-auth` `./configure` option and
add the new ones that replaced them.

Follow-up to e92edfbef6 #11490

Reviewed-by: Daniel Stenberg
Closes #12194
2023-10-25 12:45:43 +00:00
Stefan Eissing
47f5b1a37f
lib: introduce struct easy_poll_set for poll information
Connection filter had a `get_select_socks()` method, inspired by the
various `getsocks` functions involved during the lifetime of a
transfer. These, depending on transfer state (CONNECT/DO/DONE/ etc.),
return sockets to monitor and flag if this shall be done for POLLIN
and/or POLLOUT.

Due to this design, sockets and flags could only be added, not
removed. This led to problems in filters like HTTP/2 where flow control
prohibits the sending of data until the peer increases the flow
window. The general transfer loop wants to write, adds POLLOUT, the
socket is writeable but no data can be written.

This leads to cpu busy loops. To prevent that, HTTP/2 did set the
`SEND_HOLD` flag of such a blocked transfer, so the transfer loop cedes
further attempts. This works if only one such filter is involved. If a
HTTP/2 transfer goes through a HTTP/2 proxy, two filters are
setting/clearing this flag and may step on each other's toes.

Connection filters `get_select_socks()` is replaced by
`adjust_pollset()`. They get passed a `struct easy_pollset` that keeps
up to `MAX_SOCKSPEREASYHANDLE` sockets and their `POLLIN|POLLOUT`
flags. This struct is initialized in `multi_getsock()` by calling the
various `getsocks()` implementations based on transfer state, as before.

After protocol handlers/transfer loop have set the sockets and flags
they want, the `easy_pollset` is *always* passed to the filters. Filters
"higher" in the chain are called first, starting at the first
not-yet-connection one. Each filter may add sockets and/or change
flags. When all flags are removed, the socket itself is removed from the
pollset.

Example:

 * transfer wants to send, adds POLLOUT
 * http/2 filter has a flow control block, removes POLLOUT and adds
   POLLIN (it is waiting on a WINDOW_UPDATE from the server)
 * TLS filter is connected and changes nothing
 * h2-proxy filter also has a flow control block on its tunnel stream,
   removes POLLOUT and adds POLLIN also.
 * socket filter is connected and changes nothing
 * The resulting pollset is then mixed together with all other transfers
   and their pollsets, just as before.

Use of `SEND_HOLD` is no longer necessary in the filters.

All filters are adapted for the changed method. The handling in
`multi.c` has been adjusted, but its state handling the the protocol
handlers' `getsocks` method are untouched.

The most affected filters are http/2, ngtcp2, quiche and h2-proxy. TLS
filters needed to be adjusted for the connecting handshake read/write
handling.

No noticeable difference in performance was detected in local scorecard
runs.

Closes #11833
2023-10-25 09:34:32 +02:00
Daniel Stenberg
29e198bc71
tests/README: SOCKS tests are not using OpenSSH, it has its own server
Follow-up to 04fd67555c

Closes #12195
2023-10-25 09:30:30 +02:00
Jacob Hoffman-Andrews
5ecbe4df22
tets: make test documentation more user-friendly
Put the instructions to run tests right at the top of tests/README.md.

Give instructions to read the runtests.1 man page for information
about flags. Delete redundant copy of the flags documentation in the
README.

Add a mention in README.md of the important parallelism flag, to make
test runs go much faster.

Move documentation of output line format into the runtests.1 man page,
and update it with missing flags.

Fix the order of two flags in the man page.

Closes #12193
2023-10-25 07:54:54 +02:00