Commit Graph

32079 Commits

Author SHA1 Message Date
Daniel Stenberg
aba98d2f1f
buildconf.bat: remove outdated groff/nroff use
- don't try to generate the real hugehelp file, because it requires
  curl.txt which needs a build
- don't attempt to do anything in a c-ares subdirectory

Follow-up to f03c85635f
Closes #13078
2024-03-07 22:38:16 +01:00
Daniel Stenberg
5267bf52d8
http2: memory errors in the push callbacks are fatal
Use the correct nghttp2 error code accordingly.

Closes #13081
2024-03-07 22:36:49 +01:00
Viktor Szakats
e55db0c8c7
mkhelp: rename variable to fix compiler warnings
```
src\tool_operate.c(541,33): warning C4459: declaration of 'm' hides global declaration [_bld\src\curl.vcxproj]
  _bld\src\tool_hugehelp.c(8,27):
  see declaration of 'm'
src\tool_paramhlp.c(307,14): warning C4459: declaration of 'm' hides global declaration [_bld\src\curl.vcxproj]
src\tool_progress.c(118,16): warning C4459: declaration of 'm' hides global declaration [_bld\src\curl.vcxproj]
src\tool_writeout.c(288,31): warning C4459: declaration of 'm' hides global declaration [_bld\src\curl.vcxproj]
```
Ref: https://ci.appveyor.com/project/curlorg/curl/builds/49348159/job/51ee75cd2n0wj6lc#L614

Reviewed-by: Daniel Stenberg
Closes #13077
2024-03-07 19:03:30 +00:00
Daniel Stenberg
cf3b60e9aa
KNOWN_BUGS: POP3 issue when reading small chunks
Closes #12063
2024-03-07 17:40:29 +01:00
Daniel Stenberg
a03b91e725
RELEASE-NOTES: synced 2024-03-07 16:50:59 +01:00
Robert Moreton
835e4cb1d5
asyn-ares: fix data race warning
- Store the c-ares version during global init.

Prior to this change several threads could write the same data to a
static int variable at the same time. Though in practice it's not a
problem ThreadSanitizer may warn.

Reported-by: Nikita Taranov
Assisted-by: Jay Satiro

Fixes #13065
Closes #13000
2024-03-07 16:47:36 +01:00
Stefan Eissing
2c0f2e8163
hyper: implement unpausing via client reader
Just a tidy up to contain 'ifdef' pollution of common
code parts with implementation specifics.

- remove the ifdef hyper unpausing in easy.c
- add hyper client reader for CURL_CR_PROTOCOL phase
  that implements the unpause method for calling
  the hyper waker if it is set

Closes #13075
2024-03-07 15:58:30 +01:00
Stefan Eissing
8a9fbd6291
ngtcp2: no recvbuf for stream
- write response data directly to the transfer via
 `Curl_xfer_write_resp()` like we do in HTTP/2.

Closes #13073
2024-03-07 12:49:23 +01:00
Stefan Eissing
a89be3cdff
docs/cmdline-opts/.gitignore: ignore curl.txt
Closes #13076
2024-03-07 10:27:27 +01:00
Evgeny Grin (Karlson2k)
05268cf801
sha512_256: add support for GnuTLS and OpenSSL
This is a follow-up for PR #12897.

Add support for SHA-512/256 digest calculation by TLS backends.
Currently only OpenSSL and GnuTLS (actually, nettle) support
SHA-512/256.

Closes #13070
2024-03-07 10:25:11 +01:00
Evgeny Grin
1e517e9f5c
digest: add check for hashing error
Closes #13072
2024-03-07 10:24:27 +01:00
Viktor Szakats
9b81f1bef2
cmake: enable ENABLE_CURL_MANUAL by default
Meaning `curl.1` and `src/tool_hugehelp.c` are built by default,
and `--manual` in curl tool is also enabled by default.

This syncs behaviour with autotools.

For a reproducible `curl.1`, `SOURCE_DATE_EPOCH` needs to be set
to a consistent date, e.g. the timestamp of `CHANGES`.

A pre-built manual (e.g. the one distributed in the official source
tarball) will be ignored and rebuilt after this patch, unless
explicitly disabling this option.

Fixes #13028
Closes #13069
2024-03-07 09:13:36 +00:00
Stefan Eissing
deca803999
http2: push headers better cleanup
- provide common cleanup method for push headers

Closes #13054
2024-03-07 10:12:28 +01:00
Daniel Stenberg
1347cf255b
GIT-INFO: convert to markdown
Closes #13074
2024-03-07 09:43:33 +01:00
Richard Levitte
296e855d36
cmake: fix libcurl.pc and curl-config library specifications
Letting CMake figure out where libraries are located gives you full
paths. When generating libcurl.pc and curl-config, getting libraries as
full paths is unusual when one expects to get a list of -l<libname>.

To meet expectations, an effort is made to convert the full paths into
-l<libname>, possibly with -L<libdir> before it.

Fixes #6169
Fixes #12748
Closes #12930
2024-03-07 09:19:11 +01:00
Daniel Stenberg
6a13d4d75c
test463: HTTP with -d @file with file containing CR, LF and null byte 2024-03-07 08:14:45 +01:00
Daniel Stenberg
923f7f8ce5
paramhlp: fix CRLF-stripping files with "-d @file"
All CR and LF bytes should be stripped, as documented, and all other
bytes are inluded in the data. Starting now, it also excludes null bytes
as they would otherwise also cut the data short.

Reported-by: Simon K
Fixes #13063
Closes #13064
2024-03-07 08:14:45 +01:00
Viktor Szakats
ed97fe06ac
cmake: fix CURL_WINDOWS_SSPI=ON with Schannel disabled
Prior to this change `CURL_WINDOWS_SSPI` was accidentally forced `OFF`
when building without the Schannel TLS backend.

This in turn may have caused Kerberos, SPNEGO and SSPI features
disappearing even with `CURL_WINDOWS_SSPI=ON` set.

This patch fixes it by using the `CURL_USE_SCHANNEL` setting as a
default for `CURL_WINDOWS_SSPI`, but allowing a manual override.

Also update the option text to better tell its purpose.

Thanks-to: Andreas Loew
Reviewed-by: Daniel Stenberg
Ref: #13056
Closes #13061
2024-03-07 00:23:20 +00:00
Jay Satiro
2ea178ace7 KNOWN_BUGS: FTPS server compatibility on Windows with Schannel
- Remove "2.12 FTPS with Schannel times out file list operation"

- Remove "7.12 FTPS directory listing hangs on Windows with Schannel"

- Add "7.12 FTPS server compatibility on Windows with Schannel"

This change adds a more generic bug description that explains FTPS with
the latest curl and Schannel is not widely used and may have more bugs
than other TLS backends.

The two removed FTPS Schannel bugs can't be reproduced any longer and
were likely fixed by 24d6c288.

Ref: https://github.com/curl/curl/issues/5284
Ref: https://github.com/curl/curl/issues/9161
Ref: https://github.com/curl/curl/issues/12894

Closes https://github.com/curl/curl/pull/13032
2024-03-06 17:46:54 -05:00
Jay Satiro
48f54264c4 trace-config.md: remove the mutexed options list
- Remove the rendered manpage message that says:
  "[--trace-config] is mutually exclusive to --trace and -v, --verbose".

Actually it can be used with either of those options, which are mutually
exclusive to each other but not to --trace-config.

Ref: https://curl.se/docs/manpage.html#--trace-config

Closes https://github.com/curl/curl/pull/13031
2024-03-06 17:45:40 -05:00
Daniel Stenberg
62c08d5d4a
mkhelp: simplify the generated hugehelp program
Use a plain array and puts() every line, also allows us to provide the
strings without ending newlines.

- merge blank lines into the next one as a prefixed newline.
- turn eight consecutive spaces into a tab (since they can only be on the
  left side of text)
- the newly generated tool_hugehelp is 3K lines shorter and 50K smaller
- modifies the top logo layout a little by reducing the indent

Closes #13047
2024-03-06 15:56:00 +01:00
Daniel Stenberg
f03c85635f
docs: ascii version of manpage without nroff
Create ASCII version of manpage without nroff

 - build src/tool_hugegelp.c from the ascii manpage
 - move the the manpage and the ascii version build to docs/cmdline-opts
 - remove all use of nroff from the build process
 - should make the build entirely reproducible (by avoiding nroff)

 - partly reverts 2620aa9 to build libcurl option man pages one by one
   in cmake because the appveyor builds got all crazy until I did

The ASCII version of the manpage

 - is built with gen.pl, just like the manpage is
 - has a right-justified column making the appearance similar to the previous
   version
 - uses a 4-space indent per level (instead of the old version's 7)
 - does not do hyphenation of words (which nroff does)

History

  We first made the curl build use nroff for building the hugehelp file in
  December 1998, for curl 5.2.

Closes #13047
2024-03-06 15:55:59 +01:00
Stefan Eissing
9978d40ddb
lib: add void *ctx to reader/writer instances
- `struct Curl_cwriter` and `struct Curl_creader` now carry a
  `void *ctx` member that points to the instance as allocated.
- using `r->ctx` and `w->ctx` as pointer to the instance specific
  struct that has been allocated

Reported-by: Rudi Heitbaum
Fixes #13035
Closes #13059
2024-03-06 14:38:12 +01:00
Stefan Eissing
2ca530d2fa
http: fix dead code in setting post client reader
- postsize was always 0, thus the check's else never happened
  after the mime client reader was introduced

Follow-up to 0ba47146f7
Closes #13060
2024-03-06 14:35:23 +01:00
Stefan Eissing
cc6f2f0064
http2: fix push discard
- fix logic in discarding a failed pushed stream so that
  stream context is properly cleaned up

Closes #13055
2024-03-06 14:34:21 +01:00
Stefan Eissing
db5c9f4f9e
transfer.c: break receive loop in speed limited transfers
- the change breaks looping in transfer.c receive for transfers that are
  speed limited on having gotten *some* bytes.
- the overall speed limit timing is done in multi.c

Reported-by: Dmitry Karpov
Bug: https://curl.se/mail/lib-2024-03/0001.html
Closes #13050
2024-03-06 08:07:37 +01:00
Stefan Eissing
0ba47146f7
mime: add client reader
Add `mime` client reader. Encapsulates reading from mime parts, getting
their length, rewinding and unpausing.

- remove special mime handling from sendf.c and easy.c
- add general "unpause" method to client readers
- use new reader in http/imap/smtp
- make some mime functions static that are now only used internally

In addition:
- remove flag 'forbidchunk' as no longer needed

Closes #13039
2024-03-06 00:17:37 +01:00
Daniel Stenberg
6c632b216b
RELEASE-NOTES: synced 2024-03-05 17:15:27 +01:00
Daniel Stenberg
4cea0982a0
TODO: remove "build HTTP/3 with OpenSSL and nghttp3 using cmake"
Follow-up to 8e741644a2
2024-03-05 17:10:03 +01:00
Tal Regev
8e741644a2
cmake: add USE_OPENSSL_QUIC support
Closes #13034
2024-03-05 17:07:57 +01:00
Stefan Eissing
c426277b59
TIMER_STARTTRANSFER: set the same for everyone
- set TIMER_STARTTRANSFER on seeing the first response bytes
  in the download client writer, not coming from a CONNECT
- initialized the timer the same way for all protocols
- remove explicit setting of TIMER_STARTTRANSFER in file.c
  and c-hyper.c

Closes #13052
2024-03-05 17:06:47 +01:00
Michael Kaufmann
df1fcb41e2
http: better error message for HTTP/1.x response without status line
If a response without a status line is received, and the connection is
known to use HTTP/1.x (not HTTP/0.9), report the error "Invalid status
line" instead of "Received HTTP/0.9 when not allowed".

Closes #13045
2024-03-05 16:02:44 +01:00
Viktor Szakats
a54d0bdc65
KNOWN_BUGS: fix typo
Reviewed-by: Daniel Stenberg
Closes #13051
2024-03-05 12:53:50 +00:00
Sebastian Neubauer
a5dd9435ee
smpt: fix starttls
In cases where the connection was fast, curl sometimes failed to open a
connection. This fixes a regression of c2d973627b.

The regression triggered in these steps:

1. Create an smtp connection
2. Use STARTTLS
3. Receive the response
4. We are inside the loop in `smtp_statemachine`, calling
   `smtp_state_starttls_resp`
5. In the good flow, we exit the loop, re-enter `smtp_statemachine` and
   run `smtp_perform_upgrade_tls` at the start of the function.

   In the bad flow, we stay in the while loop, calling
   `Curl_pp_readresp`, which reads part of the TLS handshake and things
   go wrong.

The reason is that `Curl_pp_moredata` changed behavior and always
returns `true`, so we stay in the loop in `smtp_statemachine`. With a
slow connection `Curl_pp_readresp` cannot read new data and returns
`CURL_AGAIN`, so we leave the loop and re-enter `smtp_statemachine`.

With a fast connection, `Curl_pp_readresp` reads new data from the tcp
connection, which is part of the TLS handshake.

The fix is in `Curl_pp_moredata`, which needs to take the final line
into account and return `false` if only the final line is stored.

Closes #13048
2024-03-05 13:30:16 +01:00
Stefan Eissing
14bcea074a
lib: enhance client reader resume + rewind
- update client reader documentation
- client reader, add rewind capabilities
    - tell creader to rewind on next start
    - Curl_client_reset() will keep reader for future rewind if requested
    - add Curl_client_cleanup() for freeing all resources independent of
      rewinds
    - add Curl_client_start() to trigger rewinds
    - move rewind code from multi.c to sendf.c and make part of
      "cr-in"'s implementation
- http, move the "resume_from" handling into the client readers
    - the setup of a HTTP request is reshuffled to follow:
      * determine method, target, auth negotiation
      * install the client reader(s) for the request, including crlf
        conversions and "chunked" encoding
      * apply ranges to client reader
      * concat request headers, upgrades, cookies, etc.
      * complete request by determining Content-Length of installed
        readers in combination with method
      * send
    - add methods for client readers to
      * return the overall length they will generate (or -1 when unknown)
      * return the amount of data on the CLIENT level, so that
        expect-100 can decide if it want to apply itself
      * set a "resume_from" offset or fail if unsupported
    - struct HTTP has become largely empty now
- rename `Client_reader_*` to `Curl_creader_*`

Closes #13026
2024-03-05 13:26:05 +01:00
Viktor Szakats
9c7768cd88
openssl-quic: fix BIO leak and Windows warning
Caused by an accidentally duplicated line in
d6825df334.

```
.../lib/vquic/curl_osslq.c:1095:30: warning: implicit conversion loses integer precision: 'curl_socket_t' (aka 'unsigned long long') to 'int' [-Wshorten-64-to-32]
 1095 |   bio = BIO_new_dgram(ctx->q.sockfd, BIO_NOCLOSE);
      |         ~~~~~~~~~~~~~ ~~~~~~~^~~~~~
1 warning and 2 errors generated.
```

Reviewed-by: Stefan Eissing
Closes #13043
2024-03-05 10:54:08 +00:00
Viktor Szakats
065faf2f93
openssl-quic: fix unity build, casing, indentation
- rename static functions to avoid duplicate symbols in unity mode.
- windows -> Windows/window in error message and comment.
- fix indentation.

Reviewed-by: Stefan Eissing
Closes #13044
2024-03-05 10:54:08 +00:00
Daniel Stenberg
eb9166dc66
gen.pl: make the "manpageification" faster
The function that replaces occurances of "--longoption" with "-Z,
--longoption" etc with the proper highlight applied, no longer loops
over the options.

Closes #13041
2024-03-05 10:54:23 +01:00
Daniel Stenberg
6f685f04fd
CONTRIBUTE: update the section on documentation format
... since most of it is markdown now.

Closes #13046
2024-03-05 10:53:21 +01:00
Daniel Stenberg
07b667567f
smtp: free a temp resource
The returned address needs to be freed.

Follow-up to e3905de819
Spotted by Coverity

Closes #13038
2024-03-04 22:44:55 +01:00
Daniel Stenberg
e455490c3c
_VARIABLES.md: improve the description
Closes #13040
2024-03-04 22:43:50 +01:00
dependabot[bot]
36a95c5163 build(deps): bump fsfe/reuse-action from 2 to 3
Bumps [fsfe/reuse-action](https://github.com/fsfe/reuse-action) from 2 to 3.
- [Release notes](https://github.com/fsfe/reuse-action/releases)
- [Commits](https://github.com/fsfe/reuse-action/compare/v2...v3)

---
updated-dependencies:
- dependency-name: fsfe/reuse-action
  dependency-type: direct:production
  update-type: version-update:semver-major
...

Signed-off-by: dependabot[bot] <support@github.com>
2024-03-04 22:43:08 +01:00
Stefan Eissing
ee31f69690
pytest: adapt to API change
- pytest has changed the signature of the hook pytest_report_header()
  for some obscure reason and that change landed in our CI now

- remove the changed param that we never used anyway

Closes #13037
2024-03-04 12:47:56 +01:00
Daniel Stenberg
9454757508
cookie: if psl fails, reject the cookie
A libpsl install without data and no built-in database is now considered
bad enough to reject all cookies since they cannot be checked. It is
somewhat of a user error, but still.

Reported-by: Dan Fandrich
Closes #13033
2024-03-04 08:46:59 +01:00
Stefan Eissing
e3905de819
lib: further send/upload handling polish
- Move all the "upload_done" handling to request.c

  - add possibility to abort sending of a request
  - add `Curl_req_done_sending()` for checks
  - transfer.c: readwrite_upload() now clean

- removing data->state.ulbuf and data->req.upload_fromhere

  - as well as data->req.upload_present
  - set data->req.upload_done on having read all from
    the client and completely flushed the send buffer

- tftp, remove setting of data->req.upload_fromhere

  - serves no purpose as `upload_present` is not set
    and the data itself is directly `sendto()` anyway

- smtp, make upload EOB conversion a client reader
- xfer_ulbuf addition

  - add xfer_ulbuf for borrowing, similar to xfer_buf
  - use in file upload
  - use in c-hyper body sending

- h1-proxy, remove init of data->state.uilbuf that is never used
- smb, add own send_buf instead of using data->state.ulbuf

Closes #13010
2024-03-04 08:42:56 +01:00
Daniel Stenberg
46aea3d990
RELEASE-NOTES: synced 2024-03-04 08:19:12 +01:00
kpcyrd
ae7ad31be2
rustls: fix two warnings related to number types
Reported-by: Gisle Vanem
Follow-up to #12989
Closes #13017
2024-03-03 18:22:03 +01:00
Stefan Eissing
b1005d127f
bufq: writing into a softlimit queue cannot be partial
- when unable to obtain a new chunk on a softlimit bufq,
  this is an allocation error and needs to be reported as
  such.
- writes into a soflimit bufq never must be partial success

Reported-by: Dan Fandrich
Fixes #13020
Closes #13023
2024-03-03 17:35:47 +01:00
Dan Fandrich
ab173d1443 configure: Don't build shell completions when disabled
With the recent changes to completion file building, the files were
built always and only installation was selectively disabled.  Now, when
they are disabled they aren't even built, avoiding a build-time error in
environments where it's not possible to run the curl binary that was
just created (e.g. if library paths were not set up correctly).

Follow-up to 0f7aba83c

Reported-by: av223119 on github
Fixes #13027
Closes #13030
2024-03-02 16:36:15 -08:00
Jay Satiro
a0cbe4b867 cmdline-opts/_EXITCODES: sync with libcurl-errors
- Add error code 100 (CURLE_TOO_LARGE) to the list of error codes that
  can be returned by the curl tool.

Closes https://github.com/curl/curl/pull/13015
2024-03-02 17:59:29 -05:00