Commit Graph

31650 Commits

Author SHA1 Message Date
Daniel Stenberg
52c4d8d3f4
tool: fix the listhelp generation command
The previous command line to generate the tool_listhelp.c source file
broke with 2494b8dd51.

Make 'make listhelp' invoked in src/ generate it. Also update the
comment in the file to mention the right procedure.

Closes #12786
2024-01-25 13:54:26 +01:00
Daniel Stenberg
879a44084f
http: check for "Host:" case insensitively
When checking if the user wants to replace the header, the check should
be case insensitive.

Adding test 461 to verify

Found-by: Dan Fandrich
Ref: #12782
Closes #12784
2024-01-25 13:53:09 +01:00
Tatsuhiro Tsujikawa
fe537e21e4
configure: add libngtcp2_crypto_boringssl detection
If OpenSSL is found to be BoringSSL or AWS-LC, and ngtcp2 is requested,
try to detect libngtcp2_crypto_boringssl.

Reported-by: ウさん
Fixes #12724
Closes #12769
2024-01-25 11:27:43 +01:00
Daniel Stenberg
199c1d720d
http: remove comment reference to a removed solution
Follow-up to 58974d25d

Closes #12785
2024-01-25 11:26:23 +01:00
Stefan Eissing
1c550b17eb
pytest: Scorecard tracking CPU and RSS
Closes #12765
2024-01-25 09:16:23 +01:00
Graham Campbell
621cab26a9
GHA: bump ngtcp2, gnutls, mod_h2, quiche
- ngtcp2 to v1.2.0
- gnutls to 3.8.3
- mod_h2 to 2.0.26
- quiche to 0.20.0

Closes #12778
Closes #12779
Closes #12780
Closes #12781
2024-01-25 08:57:09 +01:00
Daniel Stenberg
f0986c6e18
ftpserver.pl: send 213 SIZE response without spurious newline 2024-01-25 08:34:23 +01:00
Daniel Stenberg
c2d973627b
pingpong: stop using the download buffer
The pingpong logic now uses its own dynbuf for receiving command
response data.

When the "final" response header for a commanad has been received, that
final line is left first in the recvbuf for the protocols to parse at
will. If there is additional data behind the final response line, the
'overflow' counter is indicate how many bytes.

Closes #12757
2024-01-25 08:34:23 +01:00
Daniel Stenberg
196074e73f
gen.pl: remove bold from .IP used for ##
Reported-by: Viktor Szakats
Fixes #12776
Closes #12777
2024-01-25 08:29:41 +01:00
Viktor Szakats
a808aab068
cmake: rework options to enable curl and libcurl docs
Rework CMake options for building/using curl tool and libcurl manuals.

- rename `ENABLE_MANUAL` to `ENABLE_CURL_MANUAL`, meaning:
  to build man page and built-in manual for curl tool.

- rename `BUILD_DOCS` to `BUILD_LIBCURL_DOCS`, meaning:
  to build man pages for libcurl.

- `BUILD_LIBCURL_DOCS` now works without having to enable
  `ENABLE_CURL_MANUAL` too.

- drop support for existing CMake-level `USE_MANUAL` option to avoid
  confusion. (It used to work with the effect of current
  `ENABLE_CURL_MANUAL`, but only by accident.)

Assisted-by: Richard Levitte
Ref: #12771
Closes #12773
2024-01-24 23:22:27 +00:00
Daniel Stenberg
162113676a
urlapi: remove assert
This assert triggers wrongly when CURLU_GUESS_SCHEME and
CURLU_NO_AUTHORITY are both set and the URL is a single path.

I think this assert has played out its role. It was introduced in a
rather big refactor.

Follow-up to 4cfa5bcc9a

Reported-by: promptfuzz_ on hackerone
Closes #12775
2024-01-24 23:15:13 +01:00
Patrick Monnerat
bc6d3bb184
tests: avoid int/size_t conversion size/sign warnings
Closes #12768
2024-01-24 15:03:05 +01:00
Daniel Stenberg
e5000e797f
GHA: add a job scanning for "bad words" in markdown
This means words, phrases or things we have decided not to use - words that
are spelled right according to the dictionary but we want to avoid. In the
name of consistency and better documentation.

Closes #12764
2024-01-24 08:44:34 +01:00
Viktor Szakats
2620aa930b
cmake: speed up curldown processing, enable by default
- cmake: enable `BUILD_DOCS` by default (this controls converting and
  installing `.3` files from `.md` sources)

- cmake: speed up generating `.3` files by using a single command per
  directory, instead of a single command per file. This reduces external
  commands by about a thousand. (There remains some CMake logic kicking
  in resulting in 500 -one per file- external `-E touch_nocreate` calls.)

- cd2nroff: add ability to process multiple input files.

- cd2nroff: add `-k` option to use the source filename to form the
  output filename. (instead of the default in-file `Title:` line.)

Follow-up to 3f08d80b22
Follow-up to ea0b575dab #12753
Follow-up to eefcc1bda4 #12730

Closes #12762
2024-01-23 23:06:39 +00:00
Richard Levitte
fe290cbadd
docs: install curl.1 with cmake as well
Closes #12759
2024-01-23 23:55:08 +01:00
Daniel Stenberg
e17b89db8f
osslq: remove the TLS library from the version output
Since we only support using a single TLS library at any one time, we
know that the TLS library for QUIC is the same that is also shown for
regular TLS.

Fixes #12763
Reported-by: Viktor Szakats
Closes #12767
2024-01-23 23:27:45 +01:00
Stefan Eissing
5125ddbb10 CI: remove unnecessary OpenSSL 3 option enable-tls1_3
.. and switch OpenSSL 3 libdir from lib64 to lib for consistency.

Closes https://github.com/curl/curl/pull/12758
2024-01-23 14:24:22 -05:00
Stefan Eissing
5f437ef3ca GHA: bump nghttp2 version to v1.59.0
- Switch to v1.59.0 for GHA CI jobs that use a specific nghttp2-version.

Closes https://github.com/curl/curl/pull/12766
2024-01-23 13:40:40 -05:00
Daniel Stenberg
38915d55f9
RELEASE-NOTES: synced 2024-01-23 14:34:36 +01:00
Daniel Stenberg
2494b8dd51
docs/cmdline: change to .md for cmdline docs
- switch all invidual files documenting command line options into .md,
   as the documentation is now markdown-looking.

 - made the parser treat 4-space indents as quotes

 - switch to building the curl.1 manpage using the "mainpage.idx" file,
   which lists the files to include to generate it, instead of using the
   previous page-footer/headers. Also, those files are now also .md
   ones, using the same format. I gave them underscore prefixes to make
   them sort separately:
   _NAME.md, _SYNOPSIS.md, _DESCRIPTION.md, _URL.md, _GLOBBING.md,
   _VARIABLES.md, _OUTPUT.md, _PROTOCOLS.md, _PROGRESS.md, _VERSION.md,
   _OPTIONS.md, _FILES.md, _ENVIRONMENT.md, _PROXYPREFIX.md,
   _EXITCODES.md, _BUGS.md, _AUTHORS.md, _WWW.md, _SEEALSO.md

 - updated test cases accordingly

Closes #12751
2024-01-23 14:30:15 +01:00
dependabot[bot]
dfe34d2559
CI: bump actions/cache from 3 to 4
Bumps [actions/cache](https://github.com/actions/cache) from 3 to 4.
- [Release notes](https://github.com/actions/cache/releases)
- [Changelog](https://github.com/actions/cache/blob/main/RELEASES.md)
- [Commits](https://github.com/actions/cache/compare/v3...v4)

---
updated-dependencies:
- dependency-name: actions/cache
  dependency-type: direct:production
  update-type: version-update:semver-major
...

Signed-off-by: dependabot[bot] <support@github.com>
Closes #12756
2024-01-23 08:39:58 +01:00
Daniel Stenberg
c28e9478cb
openssl: when verifystatus fails, remove session id from cache
To prevent that it gets used in a subsequent transfer that skips the
verifystatus check since that check can't be done when the session id is
reused.

Reported-by: Hiroki Kurosawa
Closes #12760
2024-01-23 08:26:36 +01:00
Viktor Szakats
3f08d80b22
cmake: add option to disable building docs 2024-01-23 00:29:02 +01:00
Richard Levitte
ea0b575dab
cmake: use curldown to build man pages
This throws away the previous HTML and PDF producers, to mimic what
Makefile.am does as faithfully as possible.

Closes #12753
2024-01-23 00:29:02 +01:00
Daniel Stenberg
1647017826
mksymbolsmanpage.pl: provide references to where the symbol is used 2024-01-23 00:29:02 +01:00
Daniel Stenberg
eefcc1bda4
docs: introduce "curldown" for libcurl man page format
curldown is this new file format for libcurl man pages. It is markdown
inspired with differences:

- Each file has a set of leading headers with meta-data
- Supports a small subset of markdown
- Uses .md file extensions for editors/IDE/GitHub to treat them nicely
- Generates man pages very similar to the previous ones
- Generates man pages that still convert nicely to HTML on the website
- Detects and highlights mentions of curl symbols automatically (when
  their man page section is specified)

tools:

- cd2nroff: converts from curldown to nroff man page
- nroff2cd: convert an (old) nroff man page to curldown
- cdall: convert many nroff pages to curldown versions
- cd2cd: verifies and updates a curldown to latest curldown

This setup generates .3 versions of all the curldown versions at build time.

CI:

Since the documentation is now technically markdown in the eyes of many
things, the CI runs many more tests and checks on this documentation,
including proselint, link checkers and tests that make sure we capitalize the
first letter after a period...

Closes #12730
2024-01-23 00:29:02 +01:00
Viktor Szakats
02f91d5b64
libssh2: use libssh2_session_callback_set2() with v1.11.1
To avoid a local hack to pass function pointers and to avoid
deprecation warnings when building with libssh2 v1.11.1 or newer:
```
lib/vssh/libssh2.c:3324:5: warning: 'libssh2_session_callback_set' is deprecated: since libssh2 1.11.1. Use libssh2_session_callback_set2() [-Wdeprecated-declarations]
lib/vssh/libssh2.c:3326:5: warning: 'libssh2_session_callback_set' is deprecated: since libssh2 1.11.1. Use libssh2_session_callback_set2() [-Wdeprecated-declarations]
```
Ref: https://github.com/curl/curl-for-win/actions/runs/7609484879/job/20720821100#step:3:4982

Ref: https://github.com/libssh2/libssh2/pull/1285
Ref: c0f69548be
Reviewed-by: Daniel Stenberg
Closes #12754
2024-01-22 16:10:45 +00:00
Daniel Stenberg
cdd905a985
transfer: make the select_bits_paused condition check both directions
If there is activity in a direction that is not paused, return false.

Reported-by: Sergey Bronnikov
Bug: https://curl.se/mail/lib-2024-01/0049.html
Closes #12740
2024-01-22 16:22:19 +01:00
Stefan Eissing
0535f6ec71
http3: initial support for OpenSSL 3.2 QUIC stack
- HTTP/3 for curl using OpenSSL's own QUIC stack together
  with nghttp3
- configure with `--with-openssl-quic` to enable curl to
  build this. This requires the nghttp3 library
- implementation with the following restrictions:
  * macOS has to use an unconnected UDP socket due to an
    issue in OpenSSL's datagram implementation
    See https://github.com/openssl/openssl/issues/23251
    This makes connections to non-reponsive servers hang.
  * GET requests will send the indicator that they have
    no body in a separate QUIC packet. This may result
    in processing delays or Transfer-Encodings on proxied
    requests
  * uploads that encounter blocks will use 100% cpu as
    detection of these flow control issue is not working
    (we have not figured out to pry that from OpenSSL).

Closes #12734
2024-01-22 16:15:45 +01:00
Viktor Szakats
f81a335e85
cmake: fix ENABLE_MANUAL option
Fix the `ENABLE_MANUAL` option. Set it to default to `OFF`.

Before this patch `ENABLE_MANUAL=ON` was a no-op, even though it was the
option designed to enable building and using the built-in curl manual.
(`USE_MANUAL=ON` option worked for this instead, by accident).

Ref: https://github.com/curl/curl/pull/12730#issuecomment-1902572409
Closes #12749
2024-01-22 09:34:38 +00:00
Mohammadreza Hendiani
ca01aca878
TODO: update broken link to ratelimit-headers draft
Closes #12741
2024-01-19 18:18:05 +01:00
Daniel Stenberg
ed420d9329
cmake: when USE_MANUAL=YES, build the curl.1 man page
Fixes KNOWN_BUG 15.4

Closes #12742
2024-01-19 18:14:56 +01:00
Daniel Stenberg
42be71e3ed
cmdline-opts/write-out.d: remove spurious double quotes 2024-01-19 16:37:14 +01:00
Stefan Eissing
524253dc90
rtsp: Convert assertion into debug log
Bug: https://bugs.chromium.org/p/oss-fuzz/issues/detail?id=65934

- write excess bytes to the client where the standard excess bytes
  checks will report any wrongness and fail the transfer

Fixes #12738
Closes #12739
2024-01-19 12:15:36 +01:00
Daniel Stenberg
c8cffcb8d4
headers: remove assert from Curl_headers_push
The fuzzer managed to reach the function without a terminating CR or LF
so let's handle it normally. While there, remove the goto.

Bug: https://bugs.chromium.org/p/oss-fuzz/issues/detail?id=65839

Closes #12721
2024-01-19 09:16:44 +01:00
Daniel Stenberg
48aaab55a5
curl_easy_getinfo.3: remove the wrong time value count
It said "six" time values but they are eight by now. Remove the mention
of the amount.

Closes #12727
2024-01-18 08:57:41 +01:00
Viktor Szakats
434db995a7
mbedtls: fix -Wnull-dereference and -Wredundant-decls
- Silence warning in mbedTLS v3.5.1 public headers:
  ```
  ./mbedtls/_x64-linux-musl/usr/include/psa/crypto_extra.h:489:14: warning: redundant redeclaration of 'psa_set_key_domain_parameters' [-Wredundant-decls]
  ./mbedtls/_x64-linux-musl/usr/include/psa/crypto_struct.h:354:14: note: previous declaration of 'psa_set_key_domain_parameters' was here
  ```
  Ref: ecec68a2c1
  Ref: https://github.com/libssh2/libssh2/pull/1226

- Fix compiler warnings seen with gcc 9.2.0 + cmake unity:
  ```
  ./curl/lib/vtls/mbedtls.c: In function 'mbedtls_bio_cf_read':
  ./curl/lib/vtls/mbedtls.c:189:11: warning: null pointer dereference [-Wnull-dereference]
    189 |   nread = Curl_conn_cf_recv(cf->next, data, (char *)buf, blen, &result);
        |           ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
  ./curl/lib/vtls/mbedtls.c: In function 'mbedtls_bio_cf_write':
  ./curl/lib/vtls/mbedtls.c:168:14: warning: null pointer dereference [-Wnull-dereference]
    168 |   nwritten = Curl_conn_cf_send(cf->next, data, (char *)buf, blen, &result);
        |              ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
  ```

- delete stray `#else`.

Closes #12720
2024-01-18 07:09:31 +00:00
Daniel Stenberg
6b930f1bfb
docs: cleanup nroff format use
- remove use of .BI for code snippet
- stop using .br, just do a blank line
- remove use of .PP
- remove use for .sp
- remove backslash in .IP
- use .IP instead of .TP

Closes #12731
2024-01-17 23:20:17 +01:00
Stefan Eissing
2462c9d594
test2307: fix expected failure code after ws refactoring
Fixes #12722
Closes #12728
2024-01-17 14:20:05 +01:00
Jay Satiro
7aa24f32db cf-socket: show errno in tcpkeepalive error messages
- If the socket keepalive options (TCP_KEEPIDLE, etc) cannot be set
  then show the errno in the verbose error messages.

Ref: https://github.com/curl/curl/discussions/12715#discussioncomment-8151652

Closes https://github.com/curl/curl/pull/12726
2024-01-17 03:05:45 -05:00
Jay Satiro
f4606a796e tool_getparam: stop supporting @filename style for --cookie
The `@filename` style was never documented for --cookie <data|filename>
but prior to this change curl would accept it anyway and always treat a
@ prefixed string as a filename.

That's a problem if the string also contains a = sign because then it is
documented to be interpreted as a cookie string and not a filename.

Example:

`--cookie @foo=bar`

Before: Interpreted as load cookies from filename foo=bar.

After: Interpreted as cookie `@foo=bar` (name `@foo` and value `bar`).

Other curl options with a data/filename option-value use the `@filename`
to distinguish filenames which is probably how this happened. The
--cookie option has never been documented that way.

Ref: https://curl.se/docs/manpage.html#-b

Closes https://github.com/curl/curl/pull/12645
2024-01-17 00:32:28 -05:00
Stefan Eissing
3378d2bd09
websockets: refactor decode chain
- use client writer stack for decoding frames
- move websocket protocol handler to ws.c

Closes #12713
2024-01-16 16:43:24 +01:00
Stefan Eissing
49ca84144e
websockets: check for negative payload lengths
- in en- and decoding, check the websocket frame payload lengths for
  negative values (from curl_off_t) and error the operation in that case
- add test 2307 to verify

Closes #12707
2024-01-16 14:56:15 +01:00
Daniel Stenberg
9034a16d97
docs: mention env vars not used by schannel
Ref: #12704

Co-authored-by: Jay Satiro <raysatiro@yahoo.com>

Closes #12711
2024-01-16 11:02:13 +01:00
Daniel Stenberg
ae9f01f336
tool_operate: make --remove-on-error only remove "real" files
Reported-by: Harry Sintonen
Assisted-by: Dan Fandrich

Closes #12710
2024-01-16 10:57:12 +01:00
Jay Wu
c5801a28c5
url: don't set default CA paths for Secure Transport backend
As the default for this backend is the native CA store.

Closes #12704
2024-01-16 10:48:36 +01:00
Lin Sun
4224d6e0f3
asyn-ares: with modern c-ares, use its default timeout
Closes #12703
2024-01-16 10:45:58 +01:00
Daniel Stenberg
ba01cac39b
tool_operate: stop setting the file comment on Amiga
- the URL is capped at 80 cols, which ruins it if longer
- it does not strip off URL credentials
- it is done unconditonally, not on --xattr
- we don't have Amiga in the CI which makes fixing it blindly fragile

Someone who builds and tests on Amiga can add it back correctly in a
future if there is a desire.

Reported-by: Harry Sintonen
Closes #12709
2024-01-15 18:21:16 +01:00
Stefan Eissing
036eb150d1
rtsp: deal with borked server responses
- enforce a response body length of 0, if the
  response has no Content-lenght. This is according
  to the RTSP spec.
- excess bytes in a response body are forwarded to
  the client writers which will report and fail the
  transfer

Follow-up to d7b6ce6
Fixes #12701
Closes #12706
2024-01-15 14:13:58 +01:00
Daniel Stenberg
72bd88adde
version: show only the libpsl version, not its dependencies
The libpsl version output otherwise also includes version number for its
dependencies, like IDN lib, but since libcurl does not use libpsl's IDN
functionality those components are not important.

Ref: https://github.com/curl/curl-for-win/issues/63
Closes #12700
2024-01-14 23:33:22 +01:00