Commit Graph

31922 Commits

Author SHA1 Message Date
Daniel Stenberg
e3fe020089
docs/libcurl: generate PROTOCOLS from meta-data
Remove the PROTOCOLS section from the source files completely and
instead generate them based on the header data in the curldown files.

It also generates TLS backend information for options marked for TLS as
protocol.

Closes #13175
2024-03-23 18:13:03 +01:00
Daniel Stenberg
7bc61bf469
CURLMOPT_MAX*: mention what happens if changed mid-transfer
For CURLMOPT_MAXCONNECTS and CURLMOPT_MAX_HOST_CONNECTIONS

Ref: #13158
Closes #13176
2024-03-23 11:31:36 +01:00
Daniel Stenberg
c5775007d3
docs/libcurl: add TLS backend info for all TLS options
All man pages that are listed to be for TLS now must also specify
exactly what TLS backends the option works for, or use All if they all
work.

cd2nroff makes sure this is done and that the listed backends exist.

Closes #13168
2024-03-22 14:07:55 +01:00
Daniel Stenberg
3ff3c0925d
docs/libcurl: cleanups
- CURLINFO_TLS_SESSION.md: remove mention of NSS
- CURLINFO_TLS_SSL_PTR.md: remove NSS leftover
- CURLOPT_CAINFO.md: drop mention of backends not supporting this
- CURLOPT_CAPATH.md: wolfSSL also supports this

Closes #13166
2024-03-21 22:36:27 +01:00
Daniel Stenberg
b935fd4a07
docs: make each libcurl man specify protocol(s)
The mandatory header now has a mandatory list of protocols for which the
manpage is relevant.

Most man pages already has a "PROTOCOLS" section, but this introduces a
stricter way to specify the relevant protocols.

cd2nroff verifies that at least one protocol is mentioned (which can be
`*`).

This information is not used just yet, but A) the PROTOCOLS section can
now instead get generated and get a unified wording across all manpages
and B) this allows us to more reliably filter/search for protocol
specific manpages/options.

Closes #13166
2024-03-21 15:27:06 +01:00
Stefan Eissing
98f67a6145
http2, http3: only return CURLE_PARTIAL_FILE when bytes were received
- should resolve spurious pytest failures when stream were reset
  right after response header were received

Clsoes #13151
2024-03-21 09:36:23 +01:00
Stefan Eissing
0f08b43557
http: separate response parsing from response action
- move code that triggers on end-of-response into separate function from
  parsing
- simplify some headp/headerlen usage
- add `httpversion` to SingleRequest to indicate the version of the
  current response

Closes #13134
2024-03-21 09:34:40 +01:00
Daniel Stenberg
0c820427f2
http2: remove the third (unused) argument from http2_data_done()
Closes #13154
2024-03-21 09:32:27 +01:00
Daniel Stenberg
33d9652a90
RELEASE-NOTES: synced 2024-03-21 08:28:56 +01:00
Karlson2k
e6af5652ec
RELEASE-NOTES: corrected
Corrected link for item 118

Closes #13157
2024-03-21 08:07:03 +01:00
Daniel Stenberg
fd1260d492
CURLOPT_INTERFACE.md: remove spurious amp, add see-also
Closes #13149
2024-03-19 13:54:56 +01:00
Stefan Eissing
522ea5420f
http: improve response header handling, save cpu cycles
Saving some cpu cycles in http response header processing:
- pass the length of the header line along
- use string constant sizeof() instead of strlen()
- check line length if prefix is possible
- switch on first header char to limit checks

Closes #13143
2024-03-19 07:53:43 +01:00
Daniel Stenberg
6bd4ca0ef6
tool_getparam: accept a blank -w ""
Added test 468 to verify.

Regression from 07bcae89d5 (shipped in 8.6.0)
Reported-by: Thomas Pyle
Fixes #13144
Closes #13145
2024-03-19 07:45:05 +01:00
Evgeny Grin
b6006381fb
curl_sha512_256: work around a NetBSD bug
Based on Michael Kaufmann analysis and suggestion

Closes #13133
2024-03-18 14:13:29 +01:00
Stefan Eissing
80a3b830cc
http: expect 100 rework
Move all handling of HTTP's `Expect: 100-continue` feature into a client
reader. Add sending flag `KEEP_SEND_TIMED` that triggers transfer
sending on general events like a timer.

HTTP installs a `CURL_CR_PROTOCOL` reader when announcing `Expect:
100-continue`. That reader works as follows:

- on first invocation, records time, starts the `EXPIRE_100_TIMEOUT`
  timer, disables `KEEP_SEND`, enables `KEEP_SEND_TIMER` and returns 0,
  eos=FALSE like a paused upload.

- on subsequent invocation it checks if the timer has expired. If so, it
  enables `KEEP_SEND` and switches to passing through reads to the
  underlying readers.

Transfer handling's `readwrite()` will be invoked when a timer expires
(like `EXPIRE_100_TIMEOUT`) or when data from the server arrives. Seeing
`KEEP_SEND_TIMER`, it will try to upload more data, which triggers
reading from the client readers again. Which then may lead to a new
pausing or cause the upload to start.

Flags and timestamps connected to this have been moved from
`SingleRequest` into the reader's context.

Closes #13110
2024-03-18 12:41:56 +01:00
Stefan Eissing
3d0fd382a2
mbedtls: fix pytest for newer versions
Fix the expectations in pytest for newer versions of mbedtls

Closes #13132
2024-03-18 12:37:00 +01:00
Daniel Stenberg
79cdae4fc7
ipv6.md: mention IPv4 mapped addresses
Reported-by: Josh Soref
Assisted-by: Jay Satiro
Fixes #13112
Closes #13131
2024-03-15 14:56:37 +01:00
Stefan Eissing
77b0571cdc
http: revisit http_perhapsrewind()
- use facilities provided by client readers better
- work also for non-uploading requests like GET/HEAD
- update documentation

Closes #13117
2024-03-15 10:50:44 +01:00
Stefan Eissing
6ea75877fa
test 1541: verify getinfo values on first header callback
Reported-by: chensong1211 on github
Ref: #13125
Closes #13128
2024-03-15 09:30:05 +01:00
Stefan Eissing
c765b04d11
TLS: start shutdown only when peer did not already close
- When curl sees a TCP close from the peer, do not start a TLS shutdown.
  TLS shutdown is a handshake and if the peer already closed the
  connection, it is not interested in participating.

Reported-by: dfdity on github
Assisted-by: Jiří Bok
Assisted-by: Pēteris Caune
Fixes #10290
Closes #13087
2024-03-15 09:19:58 +01:00
Daniel Stenberg
1356028f2c
RELEASE-NOTES: synced 2024-03-14 23:11:53 +01:00
Daniel Stenberg
40948189ff
curl: make --libcurl output better CURLOPT_*SSLVERSION
The option is really two enums ORed together, so it needs special
attention to make the code output nice.

Added test 1481 to verify. Both the server and the proxy versions.

Reported-by: Boris Verkhovskiy
Fixes #13127
Closes #13129
2024-03-14 17:59:22 +01:00
Daniel Stenberg
09f367977a
GHA/linux: add sysctl trick to work-around GitHub runner issue
The GitHub image runner update from 20240304.1.0 to 20240310.1
introduces a problem for clang-14. The issue is caused by
incompatibility between llvm 14 provided in ubuntu-22.04 image and the
much newer kernel configured with high-entropy ASLR.

As a work-around, we issue a sysctl command to lower the entropy and get
clang-14 to work again.

URL: https://github.com/actions/runner-images/issues/9491

Closes #13124
2024-03-14 14:04:23 +01:00
Daniel Stenberg
fb3c251ef7
SPONSORS: describe the basics
Closes #13119
2024-03-13 15:35:32 +01:00
Daniel Stenberg
0f4520adce
GOVERNANCE: document the core team
Closes #13118
2024-03-13 15:34:38 +01:00
Jay Satiro
181f5f3369 vquic-tls: fix the error code returned for bad CA file
- Return CURLE_SSL_CACERT_BADFILE if wolfSSL encounters a problem
  reading the cert file or path.

This is a follow-up to the parent commit aedbbdf1.

Reported-by: Karthikdasari0423@users.noreply.github.com

Fixes https://github.com/curl/curl/issues/13115
2024-03-13 02:03:18 -04:00
Daniel Stenberg
aedbbdf18e
vquic-tls: return appropirate errors on wolfSSL errors
Reported-by: Dexter Gerig
Closes #13107
2024-03-12 16:44:16 +01:00
Viktor Szakats
14d9afdfe7
tidy-up: one comment and EOF newlines
Reviewed-by: Daniel Stenberg
Closes #13108
2024-03-12 15:38:44 +00:00
Daniel Stenberg
e7219c2bdc
cmdline-opts: language cleanups
Use imperative mood consistently for the first sentence describing an
option.

"Set this" instead "tell curl to set" or "this sets..."

Plus some extra cleanups and rephrasing.

Closes #13106
2024-03-12 15:42:33 +01:00
Daniel Stenberg
451a7a7aa2
managen: remove space before protocols
For options that are listed for specific protocols, the protocols (shown
first within parentheses) are now output without the leading space in the
manpage output.

Closes #13105
2024-03-12 08:57:15 +01:00
Jay Satiro
942896fdd1 mbedtls: properly cleanup the thread-shared entropy
- Store the state of the thread-shared entropy for global init/cleanup.

- Use curl's thread support of mbedtls for all Windows builds instead of
  just when the threaded resolver is used via USE_THREADS_WIN32.

Prior to this change on global cleanup curl builds that have curl thread
support for mbedtls freed the entropy (8b1d2298) but failed to mark that
it had been freed, which caused problems on subsequent init + transfer.

Bug: https://github.com/curl/curl/discussions/11919#discussioncomment-8687105
Reported-by: awesomekosm@users.noreply.github.com

Closes https://github.com/curl/curl/pull/13071
2024-03-12 03:09:37 -04:00
Daniel Stenberg
3ccce37790
tool_getparam: handle non-existing (out of range) short-options
... correctly, even when they follow an existing one without a space in
between.

Verify with test 467

Follow-up to 07dd60c05b
Reported-by: Geeknik Labs
Fixes #13101
Closes #13102
2024-03-12 07:58:42 +01:00
Stefan Eissing
4e4e8af1f6
lib: move 'done' parameter to SingleRequests
A transfer may do several `SingleRequest`s for its success. This happens
regularly for authentication, follows and retries on failed connections.
The "readwrite()" calls and functions connected to those carried a `bool
*done` parameter to indicate that the current `SingleRequest` is over.
This may happen before `upload_done` or `download_done` bits of
`SingleRequest` are set.

The problem with that is now `write_resp()` protocol handlers are
invoked in places where the `bool *done` cannot be passed up to the
caller. Instead of being a bool in the call chain, it needs to become a
member of `SingleRequest`, reflecting its state.

This removes the `bool *done` parameter and adds the `done` bit to
`SingleRequest` instead. It adds `Curl_req_soft_reset()` for using a
`SingleRequest` in a follow up, clearing `done` and other
flags/counters.

Closes #13096
2024-03-11 23:27:02 +01:00
Stefan Eissing
6aeb729b5c
request: clarify message when request has been sent off
Change the "uploaded and fine" message for requests without a body

Reported-by: Karthikdasari0423 on github
Fixes #13093
Closes #13095
2024-03-11 12:02:11 +01:00
Daniel Stenberg
cd93765144
RELEASE-NOTES: synced 2024-03-11 07:30:01 +01:00
Stefan Eissing
fcef00db1a
lib: keep conn IP information together
new struct ip_quadruple for holding local/remote addr+port

- used in data->info and conn and cf-socket.c
- copy back and forth complete struct
- add 'secondary' to conn
- use secondary in reporting success for ftp 2nd connection

Reported-by: DasKutti on github
Fixes #13084
Closes #13090
2024-03-09 23:24:58 +01:00
Daniel Stenberg
1ccf1cd993
scripts/managen: the new name and home for the manpage generator
It was previously docs/cmdline-opts/gen.pl

Closes #13089
2024-03-08 17:05:08 +01:00
Daniel Stenberg
39173f66e5
VULN-DISCLOSURE-POLICY.md: update detail about CVE requests
curl is a CNA now

Closes #13088
2024-03-08 13:16:27 +01:00
Stefan Eissing
a586b8ca40
lib: client reader polish
- seek_func/seek_client, use transfer values only
    - remove copies held in `struct connectdata`, use only
      ever `data->set.seek_func`
    - resolves possible issues in multiuse connections
    - new mime post reader eliminates need to ever overwriting this

- websockets, remove empty Curl_ws_done() function

Closes #13079
2024-03-08 13:11:17 +01:00
Marcel Raad
800617fac8
lib1598: fix CURLOPT_POSTFIELDSIZE usage
It requires a `long` argument.

Closes https://github.com/curl/curl/pull/13085
2024-03-08 13:08:59 +01:00
Daniel Stenberg
71681376d8
docs/cmdline-opts: drop the curl.1 from the dist tarball
Since it is no longer needed for building tool_hugehelp.c and all the
docs is available in readable markdown format in the tarball, the peeps
that don't want to build the manpage still do good.

Removing it also fixes the complexity of out-of-tree builds when the
curl.1 exists in the source tree.
2024-03-08 08:48:32 +01:00
Daniel Stenberg
1ab2efb0ef
test1140/1173: extend wildcards to find curl.1
... in its new build path.

Also update the test scripts to be more precise in error messages to
help us understand CI errors better.

Follow-up to f03c85635f
Ref: #13029
Closes #13083
2024-03-08 08:48:30 +01:00
Daniel Stenberg
b4d73e67dc
http2: minor tweaks to optimize two struct sizes
- use BIT() instead of bool
- place the struct fields in (roughly) size order

Closes #13082
2024-03-07 23:00:45 +01:00
Daniel Stenberg
aba98d2f1f
buildconf.bat: remove outdated groff/nroff use
- don't try to generate the real hugehelp file, because it requires
  curl.txt which needs a build
- don't attempt to do anything in a c-ares subdirectory

Follow-up to f03c85635f
Closes #13078
2024-03-07 22:38:16 +01:00
Daniel Stenberg
5267bf52d8
http2: memory errors in the push callbacks are fatal
Use the correct nghttp2 error code accordingly.

Closes #13081
2024-03-07 22:36:49 +01:00
Viktor Szakats
e55db0c8c7
mkhelp: rename variable to fix compiler warnings
```
src\tool_operate.c(541,33): warning C4459: declaration of 'm' hides global declaration [_bld\src\curl.vcxproj]
  _bld\src\tool_hugehelp.c(8,27):
  see declaration of 'm'
src\tool_paramhlp.c(307,14): warning C4459: declaration of 'm' hides global declaration [_bld\src\curl.vcxproj]
src\tool_progress.c(118,16): warning C4459: declaration of 'm' hides global declaration [_bld\src\curl.vcxproj]
src\tool_writeout.c(288,31): warning C4459: declaration of 'm' hides global declaration [_bld\src\curl.vcxproj]
```
Ref: https://ci.appveyor.com/project/curlorg/curl/builds/49348159/job/51ee75cd2n0wj6lc#L614

Reviewed-by: Daniel Stenberg
Closes #13077
2024-03-07 19:03:30 +00:00
Daniel Stenberg
cf3b60e9aa
KNOWN_BUGS: POP3 issue when reading small chunks
Closes #12063
2024-03-07 17:40:29 +01:00
Daniel Stenberg
a03b91e725
RELEASE-NOTES: synced 2024-03-07 16:50:59 +01:00
Robert Moreton
835e4cb1d5
asyn-ares: fix data race warning
- Store the c-ares version during global init.

Prior to this change several threads could write the same data to a
static int variable at the same time. Though in practice it's not a
problem ThreadSanitizer may warn.

Reported-by: Nikita Taranov
Assisted-by: Jay Satiro

Fixes #13065
Closes #13000
2024-03-07 16:47:36 +01:00
Stefan Eissing
2c0f2e8163
hyper: implement unpausing via client reader
Just a tidy up to contain 'ifdef' pollution of common
code parts with implementation specifics.

- remove the ifdef hyper unpausing in easy.c
- add hyper client reader for CURL_CR_PROTOCOL phase
  that implements the unpause method for calling
  the hyper waker if it is set

Closes #13075
2024-03-07 15:58:30 +01:00