Commit Graph

29817 Commits

Author SHA1 Message Date
Stefan Eissing
ce0cad713d
openssl: test and fix for forward proxy handling (non-tunneling).
- adding pytest test_10 cases for proxy httpd setup tests
- fixing openssl bug in https: proxy hostname verification that
  used the hostname of the request and not the proxy name.

Closes #10498
2023-02-13 20:54:57 +01:00
Daniel Stenberg
17153e173d
cmdline-opts/Makefile: on error, do not leave a partial
And support 'make V=1' to show the full command line

Closes #10497
2023-02-13 20:53:45 +01:00
Daniel Stenberg
daf4e6a3a1
curl.1: make help, version and manual sections "custom"
Instead of using "multi: boolean", as these are slightly special as in
they do are not enable/disable ones.

Fixes #10490
Reported-by: u20221022 on github
Closes #10497
2023-02-13 20:52:11 +01:00
Stefan Eissing
c74bad29fd
tests: add tests for HTTP/2 and HTTP/3 to verify the header API
Test 2403 and 2503 check "header_json" output and therefore use of
header-api

Closes #10495
2023-02-13 17:14:21 +01:00
Philip H
6f20596768
CI: update wolfssl / wolfssh to 5.5.4 / 1.4.12
Closes #10493
2023-02-13 17:13:32 +01:00
Daniel Stenberg
a6ae169540
KNOW_BUGS: cleanups with some changed to TODOs
- remove "Excessive HTTP/2 packets with TCP_NODELAY"

  This is not a bug. Rather room for improvement.

I believe these have been fixed:

- 17.4 Connection failures with parallel HTTP/2
- 17.5 HTTP/2 connections through HTTPS proxy frequently stall

- remove "FTPS needs session reuse"

That is still true, but curl should also do session reuse now.

- remove "ASCII FTP"

It is documented behavior, and not single user has asked for extended
functionality here the last decade or so.

- remove "Passive transfer tries only one IP address"

add as a TODO

- remove "DoH leaks memory after followlocation"

With a recipe on how to reproduce, this is pointless to keep around

- remove "DoH does not inherit all transfer options"

add it as a TODO

Closes #10487
2023-02-13 17:10:57 +01:00
Tatsuhiro Tsujikawa
5c9ee8cef4
GHA: bump ngtcp2 workflow dependencies
Closes #10494
2023-02-13 17:07:49 +01:00
Patrick Monnerat
119fb18719
content_encoding: do not reset stage counter for each header
Test 418 verifies

Closes #10492
2023-02-13 17:06:19 +01:00
Daniel Stenberg
4858e7af1c
RELEASE-NOTES: synced 2023-02-13 11:31:28 +01:00
Jay Satiro
f438ce099b multi: stop sending empty HTTP/3 UDP datagrams on Windows
- Limit the 0-sized send procedure that is used to reset a SOCKET's
  FD_WRITE to TCP sockets only.

Prior to this change the reset was used on UDP sockets as well, but
unlike TCP sockets a 0-sized send actually sends out a datagram.

Assisted-by: Marc Hörsken

Ref: https://github.com/curl/curl/pull/9203

Fixes https://github.com/curl/curl/issues/9086
Closes https://github.com/curl/curl/pull/10430
2023-02-13 03:25:12 -05:00
Viktor Szakats
7ce140ba97
h3: silence compiler warnings
Reviewed-by: Daniel Stenberg
Fixes #10485
Closes #10486
2023-02-12 17:11:46 +00:00
Daniel Stenberg
efbf02111a
smb: return error on upload without size
The protocol needs to know the size ahead of time, this is now a known
restriction and not a bug.

Also output a clearer error if the URL path does not contain proper
share.

Ref: #7896
Closes #10484
2023-02-12 18:04:41 +01:00
Viktor Szakats
5309e32141
windows: always use curl's basename() implementation
The `basename()` [1][2] implementation provided by mingw-w64 [3] makes
assumptions about input encoding and may break with non-ASCII strings.

`basename()` was auto-detected with CMake, autotools and since
68fa9bf3f5 (2022-10-13), also in
`Makefile.mk` after syncing its behaviour with the mainline build
methods. A similar patch for curl-for-win broke official Windows
builds earlier, in release 7.83.1_4 (2022-06-15).

This patch forces all Windows builds to use curl's internal
`basename()` implementation to avoid such problems.

[1]: https://pubs.opengroup.org/onlinepubs/9699919799/utilities/basename.html
[2]: https://www.man7.org/linux/man-pages/man3/basename.3.html
[3]: https://sourceforge.net/p/mingw-w64/mingw-w64/ci/master/tree/mingw-w64-crt/misc/basename.c

Reported-by: UnicornZhang on Github
Assisted-by: Cherish98 on Github
Reviewed-by: Daniel Stenberg

Fixes #10261
Closes #10475
2023-02-12 12:52:59 +00:00
Philip H
82348603e8
Linux CI: Bump rustls-ffi to v0.9.1
Closes #10476
2023-02-12 12:36:25 +01:00
Daniel Stenberg
36fca647a4
libtest: build lib2305 with multibyte as well
Fixes a build regression.

Follow-up to 5a9a04d556
Reported-by: Viktor Szakats
Ref: https://github.com/curl/curl/pull/10475#issuecomment-1426831800

Closes #10477
2023-02-12 11:20:34 +01:00
Dmitry Atamanov
1df2dfb514 cmake: fix dev warning due to mismatched arg
The package name passed to find_package_handle_standard_args (BROTLI)
does not match the name of the calling package (Brotli). This can lead
to problems in calling code that expects find_package result variables
(e.g., _FOUND) to follow a certain pattern.

Closes https://github.com/curl/curl/pull/10471
2023-02-12 01:45:48 -05:00
James Keast
635bf5a6b2 setopt: Address undefined behaviour by checking for null
This addresses undefined behaviour found using clang's UBsan:

curl/lib/setopt.c:177:14: runtime error: applying non-zero offset 1 to null pointer
SUMMARY: UndefinedBehaviorSanitizer: undefined-behavior curl/lib/setopt.c:177:14 in

Closes #10472
2023-02-11 10:13:38 -08:00
Jacob Hoffman-Andrews
38262c944d
rustls: improve error messages
Changes numeric error codes into verbose error codes in two places.
Adds a prefix indicating that the error came from rustls, and in some
places which function it came from.

Adds special handling for RUSTLS_RESULT_UNEXPECTED_EOF, since the
default message of "Unexpected EOF" is insufficiently explanatory.

Closes #10463
2023-02-11 10:11:08 +01:00
Daniel Stenberg
0a9488c7cc
openssl: remove dead code
Follow-up to e8b00fcd6a

Due to the new 'if(!nonblocking)' check on the block a level above,
there is no need to check for it again within the same conditional.

Detected by Coverity

Closes #10473
2023-02-11 09:50:23 +01:00
Daniel Stenberg
03ce27ae2f
ngtcp2: replace removed define and stop using removed function
They were removed upstream.

Reported-by: Karthikdasari0423 on github
Fixes #10469
Closes #10474
2023-02-11 09:49:13 +01:00
Daniel Stenberg
0900997250
scripts/delta: show percent of number of files changed since last tag 2023-02-10 18:52:16 +01:00
Daniel Stenberg
251af47507
RELEASE-NOTES: synced 2023-02-10 18:49:34 +01:00
Stefan Eissing
ca95d391ea
pytest: add a test case for PUSH related things.
- checking that "103 Early Hints" are visible in curl's header dump file

Closes #10452
2023-02-10 10:30:22 +01:00
Gregory Panakkal
01772a3c39
WEBSOCKET.md: typo
Fixing missing slash for ws protocol scheme

Closes #10464
2023-02-10 10:28:40 +01:00
Stefan Eissing
c96f982166
vquic: stabilization and improvements
vquic stabilization
- udp send code shared between ngtcp2 and quiche
- quiche handling of data and events improved

ngtcp2 and pytest improvements
- fixes handling of "drain" situations, discovered in scorecard
  tests with the Caddy server.
- improvements in handling transfers that have already  data or
  are already closed to make an early return on recv

pytest
- adding caddy tests when available

scorecard improvemnts.
- using correct caddy port
- allowing tests for only httpd or caddy

Closes #10451
2023-02-10 10:26:40 +01:00
Philip H
bd12f27258
Linux CI: update some dependecies to latest tag
Closes #10458
2023-02-10 08:35:58 +01:00
Daniel Stenberg
5a9a04d556
test2305: send 3 frames, 4097 bytes each, as one message
Receive them using a 256 bytes buffer in a loop.
2023-02-10 08:28:59 +01:00
Daniel Stenberg
2e2e3d16c5
ws: fix recv of larger frames
+ remove 'oleft' from the struct
 + deal with "overflow data" in a separate dynbuf

Reported-by: Mike Duglas
Fixes #10438
Closes #10447
2023-02-10 08:28:58 +01:00
Daniel Stenberg
2aed8e179f
curl/websockets.h: extend the websocket frame struct 2023-02-10 08:28:58 +01:00
Daniel Stenberg
8360d58ac7
sws: fix typo, indentation add more ws logging 2023-02-10 08:28:58 +01:00
Daniel Stenberg
f2fa4112ad
test2304: remove stdout verification
This cripples the test somewhat but the check was bad since depending on
timing it could exit before the output was done, making the test flaky.
2023-02-10 08:28:25 +01:00
Dan Fandrich
c4095faebf CI: Add more labeler match patterns 2023-02-09 15:21:07 -08:00
Dan Fandrich
bfb9838380 CI: Retry failed downloads to reduce spurious failures
A temporary error with a remote server shouldn't cause a CI run to fail.
Also, put a cap on the time to download to fail faster on a misbehaving
server or connection and use HTTP compression where possible to reduce
download times.
2023-02-09 15:18:04 -08:00
Daniel Stenberg
33866a31c3
no-clobber.d: only use long form options in man page text
... since they are expanded and the short-form gets mentioned
automatically so if the short form is mentioned as well, it gets
repeated.

Fixes #10461
Closes #10462
Reported-by: Dan Fandrich
2023-02-09 23:38:44 +01:00
Daniel Stenberg
528583cf77
GHA: enable websockets in the torture job
Closes #10448
2023-02-09 18:54:17 +01:00
Daniel Stenberg
458b85be27
header.d: add a header file example
Closes #10455
2023-02-09 15:45:19 +01:00
Stefan Eissing
3de3ea6a64
HTTP/[23]: continue upload when state.drain is set
- as reported in #10433, HTTP/2 uploads may stall when a response is
  received before the upload is done. This happens when the
  data->state.drain is set for such a transfer, as the special handling
  in transfer.c from then on only cared about downloads.
- add continuation of uploads, if applicable, in this case.
- add pytest case test_07_12_upload_seq_large to reproduce this scenario
  (although, current nghttp2 implementation is using drain less often)

Reported-by: Lucas Pardue

Fixes #10433
Closes #10443
2023-02-09 09:13:30 +01:00
Stefan Eissing
8c762f5998
http2: minor buffer and error path fixes
- use memory buffer in full available size
- fail receive of reset/errored streams early

pytest:
- make test_05 error cases more reliable

Closes #10444
2023-02-09 08:38:01 +01:00
Federico Pellegrin
cace68e299
openldap: fix missing sasl symbols at build in specific configs
If curl is built with openldap support (USE_OPENLDAP=1) but does not
have also some other protocol (IMAP/SMTP/POP3) enabled that brings
in Curl_sasl_* functions, then the build will fail with undefined
references to various symbols:

ld: ../lib/.libs/libcurl.so: undefined reference to `Curl_sasl_decode_mech'
ld: ../lib/.libs/libcurl.so: undefined reference to `Curl_sasl_parse_url_auth_option'
ld: ../lib/.libs/libcurl.so: undefined reference to `Curl_sasl_cleanup'
ld: ../lib/.libs/libcurl.so: undefined reference to `Curl_sasl_can_authenticate'
ld: ../lib/.libs/libcurl.so: undefined reference to `Curl_sasl_continue'
ld: ../lib/.libs/libcurl.so: undefined reference to `Curl_sasl_start'
ld: ../lib/.libs/libcurl.so: undefined reference to `Curl_sasl_init'

This was tracked down to these functions bein used in openldap.c but
defined in curl_sasl.c and then forward in two vauth/ files to have
a guard against a set of #define configurations that was now extended
to cover also this case.

Example configuration targeted that could reproduce the problem:

curl 7.87.1-DEV () libcurl/7.87.1-DEV .... OpenLDAP/2.6.3
Protocols: file ftp ftps http https ldap ldaps

Closes #10445
2023-02-09 08:17:20 +01:00
Daniel Stenberg
da8e97b5d0
ws: use %Ou for outputting curl_off_t with info()
Reported-by: Mike Duglas
Fixes #10439
Closes #10441
2023-02-09 08:15:24 +01:00
Jay Satiro
b4b6e4f1fa curl_setup: Disable by default recv-before-send in Windows
Prior to this change a workaround for Windows to recv before every send
was enabled by default. The way it works is a recv is called before
every send and saves the received data, in case send fails because in
Windows apparently that can wipe out the socket's internal received
data buffer.

This feature has led to several bugs because the way libcurl operates
it waits on a socket to read or to write, and may not at all times
check for buffered receive data.

Two recent significant bugs this workaround caused:
- Broken Schannel TLS 1.3 connections (#9431)
- HTTP/2 arbitrary hangs (#10253)

The actual code remains though it is disabled by default. Though future
changes to connection filter buffering could improve the situation IMO
it's just not tenable to manage this workaround.

Ref: https://github.com/curl/curl/issues/657
Ref: https://github.com/curl/curl/pull/668
Ref: https://github.com/curl/curl/pull/720

Ref: https://github.com/curl/curl/issues/9431
Ref: https://github.com/curl/curl/issues/10253

Closes https://github.com/curl/curl/pull/10409
2023-02-09 01:30:10 -05:00
Stefan Eissing
ead2b2d4f6
http2: aggregate small SETTINGS/PRIO/WIN_UPDATE frames
add a small buffer to nghttp2 session sending in order to aggregate
small SETTINGS/PRIO/WIN_UPDATE frames that nghttp2 "writes" to the
callback individually.

Ref: #10389
Closes #10432
2023-02-08 10:41:15 +01:00
Stefan Eissing
e8b00fcd6a
openssl: store the CA after first send (ClientHello)
move Curl_ssl_setup_x509_store() call after the first send (ClientHello)
this gives time to parse CA anchors while waiting on the server reply

Ref: #10389
Closes #10432
2023-02-08 10:41:11 +01:00
Daniel Stenberg
341280e161
RELEASE-NOTES: synced 2023-02-08 09:31:29 +01:00
Anthony Hu
12e9186fc4
wolfssl: remove deprecated post-quantum algorithms
Closes #10440
2023-02-08 08:31:45 +01:00
John Bampton
0065b146eb
misc: fix spelling
Closes #10437
2023-02-08 08:30:05 +01:00
Daniel Stenberg
1695fcf57d
man pages: call the custom user pointer 'clientp' consistently
The variable had a few different names. Now try to use 'clientp'
consistently for all man pages using a custom pointer set by the
application.

Reported-by: Gerrit Renker

Fixes #10434
Closes #10435
2023-02-07 22:33:47 +01:00
Daniel Stenberg
1c91ba3939
vtls: infof using %.*s needs to provide the length as int
Fixes a Coverity warning.

Closes #10436
2023-02-07 16:33:04 +01:00
Stefan Eissing
7dad86a03f
vrls: addressing issues reported by coverity
I believe the code was secure before this, but limiting the accepted
name length to what is used in the structures should help Coverity's
analysis.

Closes #10431
2023-02-07 14:05:21 +01:00
Daniel Stenberg
95fe2bba74
tool_operate: move the 'updated' variable
This was already done by Dan Fandrich in the previous PR but somehow I
lost that fixup.

Follow-up to 349c5391f2
2023-02-07 12:16:13 +01:00