2832 Commits

Author SHA1 Message Date
Daniel Stenberg
9fc4b2c78c
tool: move tool_ftruncate64 to tool_util.c
... and the prototype to tool_setup.h, to make them both available more
widely and accurately.

Follow-up to 00bef95946d3511

Fixes #13458
Closes #13459
2024-04-25 09:20:38 +02:00
Gusted
00bef95946
tool_operate: don't truncate the etag save file by default
This fixes a regression of 75d79a4486b279100209ddf8c7fdb12955fb66e9. The
code in tool-operate truncated the etag save file, under the assumption
that the file would be written with a new etag value. However since
75d79a4486b279100209ddf8c7fdb12955fb66e9 that might not be the case
anymore and could result in the file being truncated when --etag-compare
and --etag-save was used and that the etag value matched with what the
server responded. Instead the truncation should not be done when a new
etag value should be written.

Test 3204 was added to verify that the file with the etag value doesn't
change the contents when used by --etag-compare and --etage-save and
that value matches with what the server returns on a non 2xx response.

Closes #13432
2024-04-23 11:33:20 +02:00
Daniel Stenberg
cf337d851a
tool_paramhlp: remove duplicate assign
Spotted by CodeSonar

Closes #13433
2024-04-21 10:51:12 +02:00
Daniel Stenberg
5fa594ab7b
tool_operate: init vars unconditionally in post_per_transfer
In case of (the unlikely) early return, they could otherwise remain
uninitialized

Spotted by CodeSonar

Closes #13430
2024-04-20 22:47:42 +02:00
Stephen Farrell
a362962b72
TLS: add support for ECH (Encrypted Client Hello)
An EXPERIMENTAL feature used with CURLOPT_ECH and --ech.

Closes #11922
2024-04-16 08:10:53 +02:00
Colin Leroy-Mira
bfe54b0e88
file: add support for getting basic directory listings
Not supported on Windows (yet)

Closes #13137
2024-04-11 12:37:12 +02:00
Daniel Stenberg
b4e8183f52
src/Makefile.am: access curl.txt using a relative path, not abs
... to make it work when mounted using different mount points. Like when
generated/used inside and outside of a docker image.

Closes #13320
2024-04-09 08:10:20 +02:00
Daniel Stenberg
50def7c881
NTLM_WB: drop support
The feature has not worked for months and has been marked as DEPRECATED
for six+ months.

Closes #13249
2024-04-08 13:58:58 +02:00
Daniel Stenberg
94db8e5c27
tool_getparam: output warning for leading unicode quote character
... in the option argument.

Typically this is a mistake done when copying example command lines from
online documentation using the wrong quote character.

Presumably there are also other potential quote characters that might be
used, and this check is done without even knowing that unicode is used!

Reported-by: Sanjay Pujare
Fixes #13214
Closes #13215
2024-03-31 11:59:54 +02:00
Daniel Stenberg
aba8043075
tool: follow-up getenv fix
Remove a double free. Change the IPFS env use to a plain getenv() simply
because coverity gets confused.

Follow-up to 9126b141c9398fe
Closes #13241
2024-03-31 11:04:00 +02:00
Daniel Stenberg
9126b141c9
curl: use curl_getenv instead of the curlx_ version
The curlx one was once introduced when we still considered dropping the
libcurl function at some point. To reduce confusion and to make it
easier to understand when curl_free() should be used, use the actual
libcurl function call directly instead.

Closes #13230
2024-03-30 22:45:47 +01:00
Daniel Stenberg
5564751f27
tool_xattr: "guess" URL scheme if none is provided
... when figuring out the source URL to store.

Reported-by: Dagfinn Ilmari Mannsåker
Fixes #13205
Closes #13221
2024-03-28 16:28:23 +01:00
Daniel Stenberg
a15de6f785
tool_xattr: in debug builds, act normally if CURL_FAKE_XATTR is not set
Closes #13220
2024-03-28 16:26:15 +01:00
Daniel Stenberg
2efc111ea4
curl: make --help adapt to the terminal width
Instead of assuming and working with 80 colums, try figuring out what
width is actually used.

Ref: #13141

Closes #13171
2024-03-27 14:12:26 +01:00
Daniel Stenberg
fe9f68fa61
cmdline-opts: shorter help texts
In an effort to increase the readability of the "--help all" output on
narrow (80 column) terminals.

Co-authored-by: Jay Satiro

Closes #13169
2024-03-25 13:05:24 +01:00
Daniel Stenberg
6bd4ca0ef6
tool_getparam: accept a blank -w ""
Added test 468 to verify.

Regression from 07bcae89d5d00 (shipped in 8.6.0)
Reported-by: Thomas Pyle
Fixes #13144
Closes #13145
2024-03-19 07:45:05 +01:00
Daniel Stenberg
40948189ff
curl: make --libcurl output better CURLOPT_*SSLVERSION
The option is really two enums ORed together, so it needs special
attention to make the code output nice.

Added test 1481 to verify. Both the server and the proxy versions.

Reported-by: Boris Verkhovskiy
Fixes #13127
Closes #13129
2024-03-14 17:59:22 +01:00
Daniel Stenberg
3ccce37790
tool_getparam: handle non-existing (out of range) short-options
... correctly, even when they follow an existing one without a space in
between.

Verify with test 467

Follow-up to 07dd60c05b
Reported-by: Geeknik Labs
Fixes #13101
Closes #13102
2024-03-12 07:58:42 +01:00
Daniel Stenberg
1ccf1cd993
scripts/managen: the new name and home for the manpage generator
It was previously docs/cmdline-opts/gen.pl

Closes #13089
2024-03-08 17:05:08 +01:00
Viktor Szakats
e55db0c8c7
mkhelp: rename variable to fix compiler warnings
```
src\tool_operate.c(541,33): warning C4459: declaration of 'm' hides global declaration [_bld\src\curl.vcxproj]
  _bld\src\tool_hugehelp.c(8,27):
  see declaration of 'm'
src\tool_paramhlp.c(307,14): warning C4459: declaration of 'm' hides global declaration [_bld\src\curl.vcxproj]
src\tool_progress.c(118,16): warning C4459: declaration of 'm' hides global declaration [_bld\src\curl.vcxproj]
src\tool_writeout.c(288,31): warning C4459: declaration of 'm' hides global declaration [_bld\src\curl.vcxproj]
```
Ref: https://ci.appveyor.com/project/curlorg/curl/builds/49348159/job/51ee75cd2n0wj6lc#L614

Reviewed-by: Daniel Stenberg
Closes #13077
2024-03-07 19:03:30 +00:00
Daniel Stenberg
923f7f8ce5
paramhlp: fix CRLF-stripping files with "-d @file"
All CR and LF bytes should be stripped, as documented, and all other
bytes are inluded in the data. Starting now, it also excludes null bytes
as they would otherwise also cut the data short.

Reported-by: Simon K
Fixes #13063
Closes #13064
2024-03-07 08:14:45 +01:00
Daniel Stenberg
62c08d5d4a
mkhelp: simplify the generated hugehelp program
Use a plain array and puts() every line, also allows us to provide the
strings without ending newlines.

- merge blank lines into the next one as a prefixed newline.
- turn eight consecutive spaces into a tab (since they can only be on the
  left side of text)
- the newly generated tool_hugehelp is 3K lines shorter and 50K smaller
- modifies the top logo layout a little by reducing the indent

Closes #13047
2024-03-06 15:56:00 +01:00
Daniel Stenberg
f03c85635f
docs: ascii version of manpage without nroff
Create ASCII version of manpage without nroff

 - build src/tool_hugegelp.c from the ascii manpage
 - move the the manpage and the ascii version build to docs/cmdline-opts
 - remove all use of nroff from the build process
 - should make the build entirely reproducible (by avoiding nroff)

 - partly reverts 2620aa9 to build libcurl option man pages one by one
   in cmake because the appveyor builds got all crazy until I did

The ASCII version of the manpage

 - is built with gen.pl, just like the manpage is
 - has a right-justified column making the appearance similar to the previous
   version
 - uses a 4-space indent per level (instead of the old version's 7)
 - does not do hyphenation of words (which nroff does)

History

  We first made the curl build use nroff for building the hugehelp file in
  December 1998, for curl 5.2.

Closes #13047
2024-03-06 15:55:59 +01:00
Daniel Stenberg
65405459d5
getparam: make --ftp-ssl work again
Follow-up to 9e4e527 which accidentally broke it

Reported-by: Jordan Brown
Fixes #13006
Closes #13007
2024-02-28 11:20:22 +01:00
Daniel Stenberg
6e494a2390
docs: more language cleanups
- present tense
- avoid bad words

Closes #13003
2024-02-27 16:19:43 +01:00
Daniel Stenberg
ac208ccf6f
write-out: add '%{proxy_used}'
Returns 1 if the previous transfer used a proxy, otherwise 0. Useful to
for example determine if a `NOPROXY` pattern matched the hostname or
not.

Extended test 970 and 972
2024-02-22 08:38:53 +01:00
Daniel Stenberg
ef4bd8d6c0
curl: when allocating variables, add the name into the struct
This saves the name from being an extra separate allocation.

Closes #12891
2024-02-07 23:11:40 +01:00
Daniel Stenberg
0f0edc283c
curl: exit on config file parser errors
Like when trying to import an environment variable that does not exist.

Also fix a bug for reading env variables when there is a default value
set.

Bug: https://curl.se/mail/archive-2024-02/0008.html
Reported-by: Brett Buddin

Add test 462 to verify.

Closes #12862
2024-02-07 13:45:28 +01:00
Daniel Stenberg
924e264bde tool_operate: change precedence of server Retry-After time
- When calculating the retry time, no longer allow a server's requested
  Retry-After time to take precedence over a longer retry time (either
  default algorithmic or user-specified).

Prior to this change the server's Retry-After time took precedence over
curl's retry time in all cases, but that's not always practical for
short Retry-After times depending on how busy the server is.

Bug: https://curl.se/mail/archive-2024-01/0022.html
Reported-by: Dirk Hünniger

Closes https://github.com/curl/curl/pull/12871
2024-02-07 02:49:28 -05:00
Daniel Stenberg
911fc964a1
cmdline-docs: quote and angle bracket cleanup
- make sure angle brackets are escaped
- remove a lot of superfluous double quotes
- replace several double quotes with backticks

To make nicer-looking markdown.

Closes #12884
2024-02-07 08:35:28 +01:00
Daniel Stenberg
75d79a4486
tool_cb_hdr: only parse etag + content-disposition for 2xx
... and ignore them for other response codes.

Reported-by: Harry Sintonen
Closes #12866
2024-02-05 23:40:33 +01:00
Chris Webb
541321507e
configure: add --disable-docs flag
Building man pages from curldown sources now requires perl. Add a
--disable-docs flag to configure to enable building and installing
without documentation where perl is not available or man pages are not
required. This is selected automatically (with a warning) when perl is
not found by configure.

Fixes #12832
Closes #12857
2024-02-05 19:39:52 +01:00
Daniel Stenberg
2f3e7a26bc
tool_operate: do not set CURLOPT_QUICK_EXIT in debug builds
Since it allows (small) memory leaks that interfere with torture tests
and regular memory-leak checks.

Reported-by: Dan Fandrich
Fixes #12834
Closes #12835
2024-02-01 11:43:47 +01:00
Viktor Szakats
cb343182b7
build: delete/replace 3 more clang warning pragmas
- tool_msgs: delete redundant `-Wformat-nonliteral` suppression pragma.

- whitespace formatting in `mprintf.h`, lib518, lib537.

- lib518: fix wrong variable in `sizeof()`.

- lib518: bump variables to `rlim_t`.
  Follow-up to e2b394106d543c4615a60795b7fdce04bd4e5090 #1469

- lib518: sync error message with lib537
  Follow-up to 365322b8bcf9efb6a361473d227b70f2032212ce

- lib518, lib537: replace `-Wformat-nonliteral` suppression pragmas
  by reworking test code.

Follow-up to 5b286c250829e06a135a6ba998e80beb7f43a734 #12812
Follow-up to aee4ebe59161d0a5281743f96e7738ad97fe1cd4 #12803
Follow-up to 09230127589eccc7e01c1a7217787ef8e64f3328 #12540
Follow-up to 3829759bd042c03225ae862062560f568ba1a231 #12489

Reviewed-by: Daniel Stenberg
Closes #12814
2024-01-28 23:54:32 +00:00
Viktor Szakats
5b286c2508
build: delete/replace clang warning pragmas
- delete redundant warning suppressions for `-Wformat-nonliteral`.
  This now relies on `CURL_PRINTF()` and it's theoratically possible
  that this macro isn't active but the warning is. We're ignoring this
  as a corner-case here.

- replace two pragmas with code changes to avoid the warnings.

Follow-up to aee4ebe59161d0a5281743f96e7738ad97fe1cd4 #12803
Follow-up to 09230127589eccc7e01c1a7217787ef8e64f3328 #12540
Follow-up to 3829759bd042c03225ae862062560f568ba1a231 #12489

Reviewed-by: Daniel Stenberg
Closes #12812
2024-01-27 21:19:41 +00:00
Daniel Stenberg
52c4d8d3f4
tool: fix the listhelp generation command
The previous command line to generate the tool_listhelp.c source file
broke with 2494b8dd5175cee7.

Make 'make listhelp' invoked in src/ generate it. Also update the
comment in the file to mention the right procedure.

Closes #12786
2024-01-25 13:54:26 +01:00
Viktor Szakats
a808aab068
cmake: rework options to enable curl and libcurl docs
Rework CMake options for building/using curl tool and libcurl manuals.

- rename `ENABLE_MANUAL` to `ENABLE_CURL_MANUAL`, meaning:
  to build man page and built-in manual for curl tool.

- rename `BUILD_DOCS` to `BUILD_LIBCURL_DOCS`, meaning:
  to build man pages for libcurl.

- `BUILD_LIBCURL_DOCS` now works without having to enable
  `ENABLE_CURL_MANUAL` too.

- drop support for existing CMake-level `USE_MANUAL` option to avoid
  confusion. (It used to work with the effect of current
  `ENABLE_CURL_MANUAL`, but only by accident.)

Assisted-by: Richard Levitte
Ref: #12771
Closes #12773
2024-01-24 23:22:27 +00:00
Jay Satiro
f4606a796e tool_getparam: stop supporting @filename style for --cookie
The `@filename` style was never documented for --cookie <data|filename>
but prior to this change curl would accept it anyway and always treat a
@ prefixed string as a filename.

That's a problem if the string also contains a = sign because then it is
documented to be interpreted as a cookie string and not a filename.

Example:

`--cookie @foo=bar`

Before: Interpreted as load cookies from filename foo=bar.

After: Interpreted as cookie `@foo=bar` (name `@foo` and value `bar`).

Other curl options with a data/filename option-value use the `@filename`
to distinguish filenames which is probably how this happened. The
--cookie option has never been documented that way.

Ref: https://curl.se/docs/manpage.html#-b

Closes https://github.com/curl/curl/pull/12645
2024-01-17 00:32:28 -05:00
Daniel Stenberg
ae9f01f336
tool_operate: make --remove-on-error only remove "real" files
Reported-by: Harry Sintonen
Assisted-by: Dan Fandrich

Closes #12710
2024-01-16 10:57:12 +01:00
Daniel Stenberg
ba01cac39b
tool_operate: stop setting the file comment on Amiga
- the URL is capped at 80 cols, which ruins it if longer
- it does not strip off URL credentials
- it is done unconditonally, not on --xattr
- we don't have Amiga in the CI which makes fixing it blindly fragile

Someone who builds and tests on Amiga can add it back correctly in a
future if there is a desire.

Reported-by: Harry Sintonen
Closes #12709
2024-01-15 18:21:16 +01:00
Daniel Stenberg
fb414370ac
tool_getparam: move the --rate logic into set_rate() 2024-01-08 22:39:18 +01:00
Daniel Stenberg
9e4e527735
tool_getparam: switch to an enum for every option
To make the big switch much easier to read/understand and to make it
easier to add new options.
2024-01-08 22:39:11 +01:00
Daniel Stenberg
1f4433dad4
tool_getparam: build post data using dynbuf (more) 2024-01-08 22:38:22 +01:00
Daniel Stenberg
1dba44b2f1
tool_getparam: replace malloc + copy by dynbuf for --data 2024-01-08 22:38:22 +01:00
Daniel Stenberg
f37840a46e
tool_getparam: make data_urlencode avoid direct malloc
use aprintf() instead
2024-01-08 22:38:22 +01:00
Daniel Stenberg
3870d03b5d
tool_getparam: move the --url-query logic into url_query()
This function is not doing post at all so it was always weirdly placed.
2024-01-08 22:38:22 +01:00
Daniel Stenberg
8aecfad9e6
tool_getparam: move the --data logic into set_data() 2024-01-08 22:38:22 +01:00
Daniel Stenberg
f6e97effb6
tool_getparam: unify the cmdline switch() into a single one
- easier to follow, easier to modify, easier to extend, possibly slightly
  faster

- each case now has the long option as a comment
2024-01-08 22:38:22 +01:00
Daniel Stenberg
07dd60c05b
tool_getparam: bsearch cmdline options
- the option names are now alpha sorted and lookup is a lot faster

- use case sensitive matching. It was previously case insensitive, but that
  was not documented nor tested.

- remove "partial match" feature. It was not documented, not tested and
  was always fragile as existing use could break when we add a new
  option

- lookup short options via a table

Closes #12631
2024-01-08 22:38:22 +01:00
Daniel Stenberg
07bcae89d5
tool: make parser reject blank arguments if not supported
Already in the getstr() function that clones the input argument.

Closes #12620
2024-01-03 23:21:29 +01:00