Commit Graph

32067 Commits

Author SHA1 Message Date
Stefan Eissing
deca803999
http2: push headers better cleanup
- provide common cleanup method for push headers

Closes #13054
2024-03-07 10:12:28 +01:00
Daniel Stenberg
1347cf255b
GIT-INFO: convert to markdown
Closes #13074
2024-03-07 09:43:33 +01:00
Richard Levitte
296e855d36
cmake: fix libcurl.pc and curl-config library specifications
Letting CMake figure out where libraries are located gives you full
paths. When generating libcurl.pc and curl-config, getting libraries as
full paths is unusual when one expects to get a list of -l<libname>.

To meet expectations, an effort is made to convert the full paths into
-l<libname>, possibly with -L<libdir> before it.

Fixes #6169
Fixes #12748
Closes #12930
2024-03-07 09:19:11 +01:00
Daniel Stenberg
6a13d4d75c
test463: HTTP with -d @file with file containing CR, LF and null byte 2024-03-07 08:14:45 +01:00
Daniel Stenberg
923f7f8ce5
paramhlp: fix CRLF-stripping files with "-d @file"
All CR and LF bytes should be stripped, as documented, and all other
bytes are inluded in the data. Starting now, it also excludes null bytes
as they would otherwise also cut the data short.

Reported-by: Simon K
Fixes #13063
Closes #13064
2024-03-07 08:14:45 +01:00
Viktor Szakats
ed97fe06ac
cmake: fix CURL_WINDOWS_SSPI=ON with Schannel disabled
Prior to this change `CURL_WINDOWS_SSPI` was accidentally forced `OFF`
when building without the Schannel TLS backend.

This in turn may have caused Kerberos, SPNEGO and SSPI features
disappearing even with `CURL_WINDOWS_SSPI=ON` set.

This patch fixes it by using the `CURL_USE_SCHANNEL` setting as a
default for `CURL_WINDOWS_SSPI`, but allowing a manual override.

Also update the option text to better tell its purpose.

Thanks-to: Andreas Loew
Reviewed-by: Daniel Stenberg
Ref: #13056
Closes #13061
2024-03-07 00:23:20 +00:00
Jay Satiro
2ea178ace7 KNOWN_BUGS: FTPS server compatibility on Windows with Schannel
- Remove "2.12 FTPS with Schannel times out file list operation"

- Remove "7.12 FTPS directory listing hangs on Windows with Schannel"

- Add "7.12 FTPS server compatibility on Windows with Schannel"

This change adds a more generic bug description that explains FTPS with
the latest curl and Schannel is not widely used and may have more bugs
than other TLS backends.

The two removed FTPS Schannel bugs can't be reproduced any longer and
were likely fixed by 24d6c288.

Ref: https://github.com/curl/curl/issues/5284
Ref: https://github.com/curl/curl/issues/9161
Ref: https://github.com/curl/curl/issues/12894

Closes https://github.com/curl/curl/pull/13032
2024-03-06 17:46:54 -05:00
Jay Satiro
48f54264c4 trace-config.md: remove the mutexed options list
- Remove the rendered manpage message that says:
  "[--trace-config] is mutually exclusive to --trace and -v, --verbose".

Actually it can be used with either of those options, which are mutually
exclusive to each other but not to --trace-config.

Ref: https://curl.se/docs/manpage.html#--trace-config

Closes https://github.com/curl/curl/pull/13031
2024-03-06 17:45:40 -05:00
Daniel Stenberg
62c08d5d4a
mkhelp: simplify the generated hugehelp program
Use a plain array and puts() every line, also allows us to provide the
strings without ending newlines.

- merge blank lines into the next one as a prefixed newline.
- turn eight consecutive spaces into a tab (since they can only be on the
  left side of text)
- the newly generated tool_hugehelp is 3K lines shorter and 50K smaller
- modifies the top logo layout a little by reducing the indent

Closes #13047
2024-03-06 15:56:00 +01:00
Daniel Stenberg
f03c85635f
docs: ascii version of manpage without nroff
Create ASCII version of manpage without nroff

 - build src/tool_hugegelp.c from the ascii manpage
 - move the the manpage and the ascii version build to docs/cmdline-opts
 - remove all use of nroff from the build process
 - should make the build entirely reproducible (by avoiding nroff)

 - partly reverts 2620aa9 to build libcurl option man pages one by one
   in cmake because the appveyor builds got all crazy until I did

The ASCII version of the manpage

 - is built with gen.pl, just like the manpage is
 - has a right-justified column making the appearance similar to the previous
   version
 - uses a 4-space indent per level (instead of the old version's 7)
 - does not do hyphenation of words (which nroff does)

History

  We first made the curl build use nroff for building the hugehelp file in
  December 1998, for curl 5.2.

Closes #13047
2024-03-06 15:55:59 +01:00
Stefan Eissing
9978d40ddb
lib: add void *ctx to reader/writer instances
- `struct Curl_cwriter` and `struct Curl_creader` now carry a
  `void *ctx` member that points to the instance as allocated.
- using `r->ctx` and `w->ctx` as pointer to the instance specific
  struct that has been allocated

Reported-by: Rudi Heitbaum
Fixes #13035
Closes #13059
2024-03-06 14:38:12 +01:00
Stefan Eissing
2ca530d2fa
http: fix dead code in setting post client reader
- postsize was always 0, thus the check's else never happened
  after the mime client reader was introduced

Follow-up to 0ba47146f7
Closes #13060
2024-03-06 14:35:23 +01:00
Stefan Eissing
cc6f2f0064
http2: fix push discard
- fix logic in discarding a failed pushed stream so that
  stream context is properly cleaned up

Closes #13055
2024-03-06 14:34:21 +01:00
Stefan Eissing
db5c9f4f9e
transfer.c: break receive loop in speed limited transfers
- the change breaks looping in transfer.c receive for transfers that are
  speed limited on having gotten *some* bytes.
- the overall speed limit timing is done in multi.c

Reported-by: Dmitry Karpov
Bug: https://curl.se/mail/lib-2024-03/0001.html
Closes #13050
2024-03-06 08:07:37 +01:00
Stefan Eissing
0ba47146f7
mime: add client reader
Add `mime` client reader. Encapsulates reading from mime parts, getting
their length, rewinding and unpausing.

- remove special mime handling from sendf.c and easy.c
- add general "unpause" method to client readers
- use new reader in http/imap/smtp
- make some mime functions static that are now only used internally

In addition:
- remove flag 'forbidchunk' as no longer needed

Closes #13039
2024-03-06 00:17:37 +01:00
Daniel Stenberg
6c632b216b
RELEASE-NOTES: synced 2024-03-05 17:15:27 +01:00
Daniel Stenberg
4cea0982a0
TODO: remove "build HTTP/3 with OpenSSL and nghttp3 using cmake"
Follow-up to 8e741644a2
2024-03-05 17:10:03 +01:00
Tal Regev
8e741644a2
cmake: add USE_OPENSSL_QUIC support
Closes #13034
2024-03-05 17:07:57 +01:00
Stefan Eissing
c426277b59
TIMER_STARTTRANSFER: set the same for everyone
- set TIMER_STARTTRANSFER on seeing the first response bytes
  in the download client writer, not coming from a CONNECT
- initialized the timer the same way for all protocols
- remove explicit setting of TIMER_STARTTRANSFER in file.c
  and c-hyper.c

Closes #13052
2024-03-05 17:06:47 +01:00
Michael Kaufmann
df1fcb41e2
http: better error message for HTTP/1.x response without status line
If a response without a status line is received, and the connection is
known to use HTTP/1.x (not HTTP/0.9), report the error "Invalid status
line" instead of "Received HTTP/0.9 when not allowed".

Closes #13045
2024-03-05 16:02:44 +01:00
Viktor Szakats
a54d0bdc65
KNOWN_BUGS: fix typo
Reviewed-by: Daniel Stenberg
Closes #13051
2024-03-05 12:53:50 +00:00
Sebastian Neubauer
a5dd9435ee
smpt: fix starttls
In cases where the connection was fast, curl sometimes failed to open a
connection. This fixes a regression of c2d973627b.

The regression triggered in these steps:

1. Create an smtp connection
2. Use STARTTLS
3. Receive the response
4. We are inside the loop in `smtp_statemachine`, calling
   `smtp_state_starttls_resp`
5. In the good flow, we exit the loop, re-enter `smtp_statemachine` and
   run `smtp_perform_upgrade_tls` at the start of the function.

   In the bad flow, we stay in the while loop, calling
   `Curl_pp_readresp`, which reads part of the TLS handshake and things
   go wrong.

The reason is that `Curl_pp_moredata` changed behavior and always
returns `true`, so we stay in the loop in `smtp_statemachine`. With a
slow connection `Curl_pp_readresp` cannot read new data and returns
`CURL_AGAIN`, so we leave the loop and re-enter `smtp_statemachine`.

With a fast connection, `Curl_pp_readresp` reads new data from the tcp
connection, which is part of the TLS handshake.

The fix is in `Curl_pp_moredata`, which needs to take the final line
into account and return `false` if only the final line is stored.

Closes #13048
2024-03-05 13:30:16 +01:00
Stefan Eissing
14bcea074a
lib: enhance client reader resume + rewind
- update client reader documentation
- client reader, add rewind capabilities
    - tell creader to rewind on next start
    - Curl_client_reset() will keep reader for future rewind if requested
    - add Curl_client_cleanup() for freeing all resources independent of
      rewinds
    - add Curl_client_start() to trigger rewinds
    - move rewind code from multi.c to sendf.c and make part of
      "cr-in"'s implementation
- http, move the "resume_from" handling into the client readers
    - the setup of a HTTP request is reshuffled to follow:
      * determine method, target, auth negotiation
      * install the client reader(s) for the request, including crlf
        conversions and "chunked" encoding
      * apply ranges to client reader
      * concat request headers, upgrades, cookies, etc.
      * complete request by determining Content-Length of installed
        readers in combination with method
      * send
    - add methods for client readers to
      * return the overall length they will generate (or -1 when unknown)
      * return the amount of data on the CLIENT level, so that
        expect-100 can decide if it want to apply itself
      * set a "resume_from" offset or fail if unsupported
    - struct HTTP has become largely empty now
- rename `Client_reader_*` to `Curl_creader_*`

Closes #13026
2024-03-05 13:26:05 +01:00
Viktor Szakats
9c7768cd88
openssl-quic: fix BIO leak and Windows warning
Caused by an accidentally duplicated line in
d6825df334.

```
.../lib/vquic/curl_osslq.c:1095:30: warning: implicit conversion loses integer precision: 'curl_socket_t' (aka 'unsigned long long') to 'int' [-Wshorten-64-to-32]
 1095 |   bio = BIO_new_dgram(ctx->q.sockfd, BIO_NOCLOSE);
      |         ~~~~~~~~~~~~~ ~~~~~~~^~~~~~
1 warning and 2 errors generated.
```

Reviewed-by: Stefan Eissing
Closes #13043
2024-03-05 10:54:08 +00:00
Viktor Szakats
065faf2f93
openssl-quic: fix unity build, casing, indentation
- rename static functions to avoid duplicate symbols in unity mode.
- windows -> Windows/window in error message and comment.
- fix indentation.

Reviewed-by: Stefan Eissing
Closes #13044
2024-03-05 10:54:08 +00:00
Daniel Stenberg
eb9166dc66
gen.pl: make the "manpageification" faster
The function that replaces occurances of "--longoption" with "-Z,
--longoption" etc with the proper highlight applied, no longer loops
over the options.

Closes #13041
2024-03-05 10:54:23 +01:00
Daniel Stenberg
6f685f04fd
CONTRIBUTE: update the section on documentation format
... since most of it is markdown now.

Closes #13046
2024-03-05 10:53:21 +01:00
Daniel Stenberg
07b667567f
smtp: free a temp resource
The returned address needs to be freed.

Follow-up to e3905de819
Spotted by Coverity

Closes #13038
2024-03-04 22:44:55 +01:00
Daniel Stenberg
e455490c3c
_VARIABLES.md: improve the description
Closes #13040
2024-03-04 22:43:50 +01:00
dependabot[bot]
36a95c5163 build(deps): bump fsfe/reuse-action from 2 to 3
Bumps [fsfe/reuse-action](https://github.com/fsfe/reuse-action) from 2 to 3.
- [Release notes](https://github.com/fsfe/reuse-action/releases)
- [Commits](https://github.com/fsfe/reuse-action/compare/v2...v3)

---
updated-dependencies:
- dependency-name: fsfe/reuse-action
  dependency-type: direct:production
  update-type: version-update:semver-major
...

Signed-off-by: dependabot[bot] <support@github.com>
2024-03-04 22:43:08 +01:00
Stefan Eissing
ee31f69690
pytest: adapt to API change
- pytest has changed the signature of the hook pytest_report_header()
  for some obscure reason and that change landed in our CI now

- remove the changed param that we never used anyway

Closes #13037
2024-03-04 12:47:56 +01:00
Daniel Stenberg
9454757508
cookie: if psl fails, reject the cookie
A libpsl install without data and no built-in database is now considered
bad enough to reject all cookies since they cannot be checked. It is
somewhat of a user error, but still.

Reported-by: Dan Fandrich
Closes #13033
2024-03-04 08:46:59 +01:00
Stefan Eissing
e3905de819
lib: further send/upload handling polish
- Move all the "upload_done" handling to request.c

  - add possibility to abort sending of a request
  - add `Curl_req_done_sending()` for checks
  - transfer.c: readwrite_upload() now clean

- removing data->state.ulbuf and data->req.upload_fromhere

  - as well as data->req.upload_present
  - set data->req.upload_done on having read all from
    the client and completely flushed the send buffer

- tftp, remove setting of data->req.upload_fromhere

  - serves no purpose as `upload_present` is not set
    and the data itself is directly `sendto()` anyway

- smtp, make upload EOB conversion a client reader
- xfer_ulbuf addition

  - add xfer_ulbuf for borrowing, similar to xfer_buf
  - use in file upload
  - use in c-hyper body sending

- h1-proxy, remove init of data->state.uilbuf that is never used
- smb, add own send_buf instead of using data->state.ulbuf

Closes #13010
2024-03-04 08:42:56 +01:00
Daniel Stenberg
46aea3d990
RELEASE-NOTES: synced 2024-03-04 08:19:12 +01:00
kpcyrd
ae7ad31be2
rustls: fix two warnings related to number types
Reported-by: Gisle Vanem
Follow-up to #12989
Closes #13017
2024-03-03 18:22:03 +01:00
Stefan Eissing
b1005d127f
bufq: writing into a softlimit queue cannot be partial
- when unable to obtain a new chunk on a softlimit bufq,
  this is an allocation error and needs to be reported as
  such.
- writes into a soflimit bufq never must be partial success

Reported-by: Dan Fandrich
Fixes #13020
Closes #13023
2024-03-03 17:35:47 +01:00
Dan Fandrich
ab173d1443 configure: Don't build shell completions when disabled
With the recent changes to completion file building, the files were
built always and only installation was selectively disabled.  Now, when
they are disabled they aren't even built, avoiding a build-time error in
environments where it's not possible to run the curl binary that was
just created (e.g. if library paths were not set up correctly).

Follow-up to 0f7aba83c

Reported-by: av223119 on github
Fixes #13027
Closes #13030
2024-03-02 16:36:15 -08:00
Jay Satiro
a0cbe4b867 cmdline-opts/_EXITCODES: sync with libcurl-errors
- Add error code 100 (CURLE_TOO_LARGE) to the list of error codes that
  can be returned by the curl tool.

Closes https://github.com/curl/curl/pull/13015
2024-03-02 17:59:29 -05:00
Stefan Eissing
9e2ee70494 hyper: disable test1598 due to lack of trailer support
Follow-up to 50838095

Closes #13016
2024-03-01 01:35:55 -08:00
Dan Fandrich
ddb87160f5 ftp: Mark a const buffer as const 2024-03-01 01:16:24 -08:00
Dan Fandrich
dff74ae8bb appveyor: Properly skip if only CircleCI is changed 2024-03-01 01:16:24 -08:00
Dan Fandrich
57777a262a docs: Update minimal binary size in INSTALL.md
Include more options to reduce binary size.
2024-03-01 01:16:24 -08:00
Dan Fandrich
0f7aba83cc configure: Don't make shell completions without perl
The code that attempted to skip building the shell completions didn't
work properly and tried to build them even if perl wasn't available.
This step, as well as the install step, is now properly skipped without
perl.

Follow-up to 89733e2dd

Closes #13022
2024-03-01 01:16:24 -08:00
RainRat
2cd78f525c misc: Fix typos in docs and lib
This fixes miscellaneous typos and duplicated words in the docs, lib
and test comments and a few user facing errorstrings.

Author: RainRat on Github
Reviewed-by: Daniel Gustafsson <daniel@yesql.se>
Reviewed-by: Dan Fandrich <dan@coneharvesters.com>
Closes: #13019
2024-03-01 09:59:48 +01:00
Dan Fandrich
89733e2dd2 configure: build & install shell completions when enabled
The --with-fish-functions-dir and --with-zsh-functions-dir options
currently have no effect on a normal build because the scripts/ directory
where they're used is not built. Add scripts/ to a normal build and
change the completion options to default to off to preserve the existing
behaviour.

Closes: #12906
2024-02-29 16:41:31 -08:00
Dan Fandrich
dcf382453f github/labeler: improve the match patterns 2024-02-29 13:04:23 -08:00
Stefan Eissing
5083809529
tests: add test1598 for POST with trailers
- test POST fields with trailers and chunked encoding

Ref: #12938
Closes #13009
2024-02-28 23:03:02 +01:00
Daniel Stenberg
32e0544dc8
cmdline-opts/_VERSION: provide %VERSION correctly
... so that it does not get included verbatim in the output. Fixes a
regression shipped in 8.6.0.

Also fix a format mistake in form.md

Closes #13008
2024-02-28 15:53:12 +01:00
Stefan Eissing
9369c30cd8
lib: Curl_read/Curl_write clarifications
- replace `Curl_read()`, `Curl_write()` and `Curl_nwrite()` to
  clarify when and at what level they operate
- send/recv of transfer related data is now done via
  `Curl_xfer_send()/Curl_xfer_recv()` which no longer has
  socket/socketindex as parameter. It decides on the transfer
  setup of `conn->sockfd` and `conn->writesockfd` on which
  connection filter chain to operate.
- send/recv on a specific connection filter chain is done via
  `Curl_conn_send()/Curl_conn_recv()` which get the socket index
  as parameter.
- rename `Curl_setup_transfer()` to `Curl_xfer_setup()` for
  naming consistency
- clarify that the special CURLE_AGAIN hangling to return
  `CURLE_OK` with length 0 only applies to `Curl_xfer_send()`
  and CURLE_AGAIN is returned by all other send() variants.
- fix a bug in websocket `curl_ws_recv()` that mixed up data
  when it arrived in more than a single chunk (to be made
  into a sperate PR, also)

Added as documented [in
CLIENT-READER.md](5b1f31dfba/docs/CLIENT-READERS.md).

- old `Curl_buffer_send()` completely replaced by new `Curl_req_send()`
- old `Curl_fillreadbuffer()` replaced with `Curl_client_read()`
- HTTP chunked uploads are now formatted in a client reader added when
  needed.
- FTP line-end conversions are done in a client reader added when
  needed.
- when sending requests headers, remaining buffer space is filled with
  body data for sending in "one go". This is independent of the request
  body size. Resolves #12938 as now small and large requests have the
  same code path.

Changes done to test cases:

- test513: now fails before sending request headers as this initial
  "client read" triggers the setup fault. Behaves now the same as in
  hyper build
- test547, test555, test1620: fix the length check in the lib code to
  only fail for reads *smaller* than expected. This was a bug in the
  test code that never triggered in the old implementation.

Closes #12969
2024-02-28 12:58:55 +01:00
Daniel Gustafsson
8d67c61c47 curldown: Fix email address in Copyright
The curldown conversion accidentally replaced daniel@haxx.se with
just daniel.se.  This reverts back to the proper email address in
the curldown docs as well as in a few other stray places where it
was incorrect (while unrelated to curldown).

Reviewed-by: Daniel Stenberg <daniel@haxx.se>
Closes: #12997
2024-02-28 11:28:10 +01:00