Commit Graph

29964 Commits

Author SHA1 Message Date
Daniel Stenberg
05616379ee
headers: make curl_easy_header and nextheader return different buffers
By letting curl_easy_header() and curl_easy_nextheader() store the
header data in their own struct storage when they return a pointer to
it, it makes it possible for applications to use them both in a loop.
Like the curl tool does.

Reported-by: Boris Okunskiy
Fixes #10704
Closes #10707
2023-03-09 00:13:41 +01:00
rcombs
b1d735956f
urlapi: take const args in _dup and _get functions
Closes #10708
2023-03-08 15:38:26 +01:00
rcombs
95cb7d3166
urlapi: avoid mutating internals in getter routine
This was not intended.

Closes #10708
2023-03-08 15:38:18 +01:00
Daniel Stenberg
0a0c9b6dfa
urlapi: '%' is illegal in host names
Update test 1560 to verify

Ref: #10708
Closes #10711
2023-03-08 15:33:43 +01:00
Daniel Stenberg
0546ed54c2
ftp: make the 'ftpauth' a more normal 'char *'-array
Closes #10703
2023-03-08 15:32:16 +01:00
Evgeny Grin (Karlson2k)
372b95f77f
doc: fix compiler warning in libcurl.m4
Current test for curl_free() may produce warnings with strict compiler
flags or even with default compiler flags with upcoming versions.
These warning could turned into errors by -Werror or similar flags.
Such warnings/errors are avoided by this patch.

Closes #10710
2023-03-08 14:24:08 +01:00
Viktor Szakats
75087c3baa
misc: fix typos
Closes #10706
2023-03-08 08:00:35 +00:00
Stefan Eissing
a26418cf14
ftp: active mode with SSL, add the damn filter
- since 7.87.0 we lost adding the SSL filter for an active
  FTP connection that uses SSL. This leads to hangers and timeouts
  as reported in #10666.

Reported-by: SandakovMM on github
Fixes #10666
Closes #10669
2023-03-07 15:22:22 +01:00
Daniel Stenberg
2a31086f39
docs: extend the URL API descriptions
Closes #10701
2023-03-07 13:36:10 +01:00
Stefan Eissing
7c5637b8b4
url: fix logic in connection reuse to deny reuse on "unclean" connections
- add parameter to `conn_is_alive()` cfilter method that returns
  if there is input data waiting on the connection
- refrain from re-using connnection from the cache that have
  input pending
- adapt http/2 and http/3 alive checks to digest pending input
  to check the connection state
- remove check_cxn method from openssl as that was just doing
  what the socket filter now does.
- add tests for connection reuse with special server configs

Closes #10690
2023-03-07 12:50:31 +01:00
Daniel Stenberg
6466071e8e
x509asn1: use plain %x, not %lx, when the arg is an int
Pointed out by Coverity.

Closes #10689
2023-03-06 23:59:12 +01:00
Stefan Eissing
48cd032623
http2: fix handling of RST and GOAWAY to recognize partial transfers
- a reset transfer (HTTP/2 RST) did not always lead to the proper
  error message on receiving its response, leading to wrong reports
  of a successful transfer
- test_05_02 was able to trigger this condition with increased transfer
  count. The simulated response errors did not carry a 'Content-Length'
  so only proper RST handling could detect the abort
- When doing such transfers in parallel, a connection could enter the
  state where
  a) it had been closed (GOAWAY received)
  b) the RST had not been "seen" for the transfer yet
  or c) the GOAWAY announced an error and the last successful
  stream id was not checked against ongoing transfers

Closes #10693
2023-03-06 23:58:24 +01:00
Stefan Eissing
b0564c1d54
tests: use dynamic ports numbers in pytest suite
- necessary ports are bound at start of test suite and then
  given to server fixtures for use.
- this make parallel use of pytest (in separate directories),
  practically safe for use as OS tend to not reuse such port numbers
  for a while

Closes #10692
2023-03-06 23:44:45 +01:00
Stefan Eissing
257416023d
connect: fix time_connect and time_appconnect timer statistics
- time_connect was not updated when the overall connection failed,
  e.g. when SSL verification was unsuccessful, refs #10670
- rework gather those values to interrogate involved filters,
  also from all eyeballing attempts, to report the maximum of
  those values.
- added 3 test cases in test_06 to check reported values on
  successful, partially failed and totally failed connections.

Reported-by: Master Inspire
Fixes #10670
Closes #10671
2023-03-06 20:16:43 +01:00
Daniel Stenberg
93eefa6ba1
test1905: update output cookie order
After the #10685 update
2023-03-06 15:52:00 +01:00
Daniel Stenberg
076764c5d7
test420: verify expiring cookies
Cookies that are loaded fine from a jar but then are expired in headers.
2023-03-06 15:52:00 +01:00
Daniel Stenberg
45d4bf5af3
cookie: don't load cookies again when flushing
Reported-by: Sergio Mijatovic
Fixes #10677
Closes #10685
2023-03-06 15:51:56 +01:00
Daniel Stenberg
c8fe72a37e
RELEASE-NOTES: synced 2023-03-06 15:48:54 +01:00
andy5995
7c3822fe45
docs: note '--data-urlencode' option
Closes #10687
2023-03-06 15:35:48 +01:00
Daniel Stenberg
e4d5685cb5
DEPRECATE: the original legacy mingw version 1
Remove completely in September 2023

Closes #10667
2023-03-06 11:31:33 +01:00
Harry Sintonen
755ddbe901
rand: use arc4random as fallback when available
Normally curl uses cryptographically strong random provided by the
selected SSL backend. If compiled without SSL support, a naive built-in
function was used instead.

Generally this was okay, but it will result in some downsides for non-
SSL builds, such as predictable temporary file names.

This change ensures that arc4random will be used instead, if available.

Closes #10672
2023-03-06 11:21:38 +01:00
Grisha Levit
1dd929674e
tool: dump headers even if file is write-only
The fixes in #10079 brought a (seemingly unrelated) change of open mode
from `wb`/`ab` to `wb+`/`ab+` for the headerfile. This makes it no
longer possible to write the header file to e.g. a pipe, like:

    curl -D >(grep ...) file:///dev/null

Which presently results in `Warning: Failed to open /dev/fd/63`

See #10079
Closes #10675
2023-03-06 11:14:37 +01:00
Jay Satiro
c12e8bfa89 tests: fix gnutls-serv check
- If gnutls-serv doesn't exist then don't try to execute it.

Follow-up to 2fdc1d81.

Closes https://github.com/curl/curl/pull/10688
2023-03-06 03:52:46 -05:00
Daniel Stenberg
54605666ed
lib1560: fix enumerated type mixed with another type
Follow-up to c84c0f9aa3

Closes #10684
2023-03-06 08:14:42 +01:00
Viktor Szakats
8cfc936f5c
cmake: fix enabling LDAPS on Windows
Before this patch, enabling LDAPS required a manual C flag:
c1cfc31cfc/curl-cmake.sh (L105)

Fix this and enable LDAPS automatically when using `wldap32` (and
when not explicitly disabled). This matches autotools and `Makefile.mk`
behavior. Also remove issue from KNOWN_BUGS.

Add workaround for MSVS 2010 warning triggered by LDAPS now enabled
in more CI tests:
`ldap.c(360): warning C4306: 'type cast' : conversion from 'int' to 'void *' of greater size`
Ref: https://ci.appveyor.com/project/curlorg/curl/builds/46408284/job/v8mwl9yfbmoeqwlr#L312

Reported-by: JackBoosY on github
Reviewed-by: Jay Satiro
Reviewed-by: Marcel Raad
Fixes #6284
Closes #10674
2023-03-05 19:55:14 +00:00
Viktor Szakats
8d0fa3a70d
Makefile.mk: delete redundant HAVE_LDAP_SSL macro [ci skip]
Since abebb2b893, we set this macro for
all Windows `wldap32` builds using `Makefile.mk`.

For OpenLDAP builds this macro is not enough to enable LDAPS, and
OpenLDAP is not an option in `Makefile.mk`. For Novell LDAP it might
have helped, but it's also not an option anymore in `Makefile.mk`.

The future for LDAPS is that we should enable it by default without
extra build knobs.

Reviewed-by: Marcel Raad
Closes #10681
2023-03-05 19:52:21 +00:00
Viktor Szakats
016c62c4d5
cmake: skip CA-path/bundle auto-detection in cross-builds
Also remove issue from KNOWN_BUGS.

Reported-by: Cristian Morales Vega
Reviewed-by: Marcel Raad
Fixes #6178
Closes #10676
2023-03-05 19:51:52 +00:00
Daniel Stenberg
e135bc9d31
schannel: loop over the algos to pick the selected one
Avoid using the funny macro and the extra buffer copy.

Closes #10647
2023-03-03 23:26:41 +01:00
Daniel Stenberg
56d5c07bbb
wildcard: remove files and move functions into ftplistparser.c 2023-03-03 23:25:30 +01:00
Daniel Stenberg
9c188e771c
ftp: allocate the wildcard struct on demand
The feature is rarely used so this frees up data for the vast majority
of easy handles that don't use it.

Rename "protdata" to "ftpwc" since it is always an FTP wildcard struct
pointer. Made the state struct field an unsigned char to save space.

Closes #10639
2023-03-03 23:25:23 +01:00
Daniel Stenberg
c84c0f9aa3
lib1560: test parsing URLs with ridiculously large fields
In the order of 120K.

Closes #10665
2023-03-03 23:23:53 +01:00
Brad Spencer
ad4997e5b2
urlapi: parse IPv6 literals without ENABLE_IPV6
This makes the URL parser API stable and working the same way
independently of libcurl supporting IPv6 transfers or not.

Closes #10660
2023-03-03 10:05:08 +01:00
Jan Engelhardt
73e9e6d767
build: drop the use of XC_AMEND_DISTCLEAN
Because automake used to delete depdirs at once (.deps) and there was an issue
with portability, curl's XC_AMEND_DISTCLEAN greps the Makefiles in an attempt
to build a list of all depfiles and delete them individually instead.

Since commit 08849db866b44510f6b8fd49e313c91a43a3dfd3, automake switched from
deleting directories to individual files. curl's custom logic now finds a lot
more results with the grep (the filtering of these results isn't great), which
causes a massive bloating of the Makefile in the order of O(n^2).

Also remove now-unused XC_AMEND_DISTCLEAN macro group

References: https://github.com/curl/curl/issues/9843
References: https://debbugs.gnu.org/cgi/bugreport.cgi?bug=59288

Reported-by: Ilmari Lauhakangas
Fixes #9843
Closes #10661
2023-03-03 09:54:44 +01:00
Balakrishnan Balasubramanian
351ea4ff53
test1470: test socks proxy using unix sockets and connect to https
Similar to test1468 except using https instead of http

Closes #10662
2023-03-03 09:18:21 +01:00
Daniel Stenberg
e4dfe6fc2f
test1960: verify CURL_SOCKOPT_ALREADY_CONNECTED
When returned from the CURLOPT_SOCKOPTFUNCTION, like when we have a
custom socket connected in the app, passed in to libcurl.

Verifies the fix in #10648

Closes #10651
2023-03-03 08:36:25 +01:00
Stefan Eissing
e497a96a0e
tests: rename tests/tests-httpd to tests/http
- httpd is only one server we test with
 - the suite coveres the HTTP protocol in general where
   the default test cases need a more beefy environment

Closes #10654
2023-03-02 14:05:19 +01:00
Stefan Eissing
9fd2d5aa72
socket: detect "dead" connections better, e.g. not fit for reuse
- refs #10646 where reuse was attempted on closed connections in the
  cache, leading to an exhaustion of retries on a transfer
- the mistake was that poll events like POLLHUP, POLLERR, etc
  were regarded as "not dead".
- change cf-socket filter check to regard such events as inidication
  of corpsiness.
- vtls filter checks: fixed interpretation of backend check result
  when inconclusive to interrogate status further down the filter
  chain.

Reported-by: SendSonS on github
Fixes #10646
Closes #10652
2023-03-02 09:03:06 +01:00
Stefan Eissing
9deebd311b
lib: give source files cf-http.* better fitting names
Closes #10656
2023-03-02 08:48:34 +01:00
Stefan Eissing
8151d7babc http2: fix code indent
Closes https://github.com/curl/curl/pull/10655
2023-03-02 00:09:53 -05:00
Shankar Jadhavar
233810bb5f
cf-socket: if socket is already connected, return CURLE_OK
In 7.87.0, if callback method for CURLOPT_SOCKOPTFUNCTION returns
CURL_SOCKOPT_ALREADY_CONNECTED then curl library used to return
CURLE_OK.  n 7.88.0, now even if callback returns
CURL_SOCKOPT_ALREADY_CONNECTED, curl library still tries to connect to
socket by invoking method do_connect().

This is regression caused by commit
https://github.com/curl/curl/commit/71b7e0161032927cdfb

Fix: Check if we are already connected and return CURLE_OK.

Fixes #10626
Closes #10648
2023-03-01 11:52:18 +01:00
Jay Satiro
d36c632c86 DYNBUF.md: note Curl_dyn_add* calls Curl_dyn_free on failure
This is the existing behavior and it has been widely assumed in the
codebase.

Closes https://github.com/curl/curl/pull/10645
2023-03-01 04:18:15 -05:00
Stefan Eissing
d9ccc75b00 http2: fix upload busy loop
- Set KEEP_SEND_PAUSE when exhausting remote HTTP/2 window size of a
  stream.

- Clear KEEP_SEND_PAUSE when receiving HTTP/2 window updates on a paused
  stream.

- Also fix http2 send compiler warnings reported in #10449.

Prior to this change, starting in 71b7e016 which precedes 7.88.0,
libcurl may eat CPU during HTTP/2 upload.

Reported-by: Jay Satiro

Fixes https://github.com/curl/curl/issues/10449
Fixes https://github.com/curl/curl/issues/10618
Closes https://github.com/curl/curl/pull/10627
2023-03-01 04:13:49 -05:00
Daniel Stenberg
c50a6eee04
sectransp: make read_cert() use a dynbuf when loading
Closes #10632
2023-03-01 09:38:43 +01:00
Jay Satiro
0b84d0cf1b transfer: limit Windows SO_SNDBUF updates to once a second
- Change readwrite_upload() to call win_update_buffer_size() no more
  than once a second to update SO_SNDBUF (send buffer limit).

Prior to this change during an upload readwrite_upload() could call
win_update_buffer_size() anywhere from hundreds of times per second to
an extreme test case of 100k per second (which is likely due to a bug,
see #10618). In the latter case WPA profiler showed
win_update_buffer_size was the highest capture count in
readwrite_upload. In any case the calls were excessive and unnecessary.

Ref: https://github.com/curl/curl/pull/2762

Closes https://github.com/curl/curl/pull/10611
2023-03-01 01:15:28 -05:00
Daniel Stenberg
a496d46a7b
RELEASE-NOTES: synced 2023-02-28 23:08:14 +01:00
Stefan Eissing
cbe9d82c22
http2: fix for http2-prior-knowledge when reusing connections
- refs #10634 where errors in the HTTP/2 framing layer are observed.
- the bug was that on connection reuse, the code attempted to switch
  in yet another layer of HTTP/2 handling instead of detecting that
  this was already in place.
- added pytest testcase reproducing the issue.

Reported-by: rwmjones on github
Fixes #10634
Closes #10643
2023-02-28 23:04:00 +01:00
Stefan Eissing
efa1aba97c
cf-socket: fix handling of remote addr for accepted tcp sockets
- do not try to determine the remote address of a listen socket. There
  is none.
- Update remote address of an accepted socket by getpeername() if
  available.

Reported-by: Harry Sintonen
Fixes #10622
Closes #10642
2023-02-28 23:03:17 +01:00
Stefan Eissing
a4d015e69f
http: fix unix domain socket use in https connects
- when h2/h3 eyeballing was involved, unix domain socket
  configurations were not honoured
- configuring --unix-socket will disable HTTP/3 as candidate for eyeballing
- combinatino of --unix-socket and --http3-only will fail during initialisation
- adding pytest test_11 to reproduce

Reported-by: Jelle van der Waa
Fixes #10633
Closes #10641
2023-02-28 22:59:19 +01:00
Daniel Stenberg
c9c3ec482b
setopt: move the CURLOPT_CHUNK_DATA pointer to the set struct
To make duphandle work etc

Closes #10635
2023-02-28 08:44:28 +01:00
Viktor Szakats
51211a31a5
quic/schannel: fix compiler warnings
Fixes #10603
Closes #10616
2023-02-28 08:43:16 +01:00