Commit Graph

2540 Commits

Author SHA1 Message Date
Daniel Gustafsson
bf327a9dfe macos9: remove partial support
The support for compiling on Mac OS 9 hasn't been modified since 2001
and has no active maintainer or packager, so it's time to remove it as
it's incredibly unlikely to work. If a maintainer re-emerges it can be
resurrected from Git history.

Closes: #8836
Reviewed-by: Daniel Stenberg <daniel@haxx.se>
2022-05-16 10:10:16 +02:00
Daniel Stenberg
0356804d13
tool_operate: make sure --fail-with-body works with --retry
... in the same way --fail already does.

Reported-by: Jakub Bochenski
Fixes #8845
Closes #8847
2022-05-16 10:04:06 +02:00
Daniel Stenberg
8c7ee9083d
post_per_transfer: remove the updated file name
When --remove-on-error is used with --no-clobber, it might have an
updated file name to remove.

Bug: https://curl.se/docs/CVE-2022-27778.html

CVE-2022-27778

Reported-by: Harry Sintonen

Closes #8824
2022-05-09 12:56:21 +02:00
Daniel Gustafsson
79bf349a50 tool_getparam: simplify conditional statement
param_place cannot be NULL here since we immediately efter this block
perform arithmetic on it (and use it in order to get here) so there is
little reason to check.

Closes: #8786
Reviewed-by: Daniel Stenberg <daniel@haxx.se>
2022-05-02 14:04:59 +02:00
Daniel Gustafsson
a7b29122cb curl: free resource in error path
If the new filename cannot be generated due to memory pressure, free
the allocated aname on the way out to avoid a small leak.

Closes: #8770
Reviewed-by: Daniel Stenberg <daniel@haxx.se>
2022-04-30 21:25:15 +02:00
Daniel Gustafsson
6d86193377 curl: guard against size_t wraparound in no-clobber code
When generating the new filename, make sure we aren't overflowing the
size_t limit when calculating the new length. This is mostly academic
but good code hygeine nonetheless.

Closes: #8771
Reviewed-by: Daniel Stenberg <daniel@haxx.se>
2022-04-30 21:17:40 +02:00
Daniel Stenberg
a09a12806e
tool_getparam: error out on missing -K file
Add test 411 to verify.

Reported-by: Median Median Stride
Bug: https://hackerone.com/reports/1542881
Closes #8731
2022-04-25 11:38:50 +02:00
Daniel Stenberg
4ec0549c39
misc: update copyright year ranges 2022-04-25 09:31:09 +02:00
Daniel Stenberg
60452236fa
c_escape: escape '?' in generated --libcurl code
In order to avoid the risk of it being used in an accidental trigraph in
the generated code.

Reported-by: Harry Sintonen
Bug: https://hackerone.com/reports/1548535
Closes #8742
2022-04-25 00:29:22 +02:00
Marc Hoersken
68035af29d
timediff.[ch]: add curlx helper functions for timeval conversions
Also move timediff_t definitions from timeval.h to timediff.h and
then make timeval.h include the new standalone-capable timediff.h.

Reviewed-by: Jay Satiro
Reviewed-by: Daniel Stenberg

Supersedes #5888
Closes #8595
2022-04-17 13:11:27 +02:00
Emanuele Torre
77a6bf8489
tool_paramhlp: use feof(3) to identify EOF correctly when using fread(3)
This loop was using the number of bytes read from the file as condition
to keep reading.

From Linux's fread(3) man page:
> On success, fread() and fwrite() return the number of items read or
> written. This number equals the number of bytes transferred only when
> size is 1. If an error occurs, or the end of the file is reached, the
> return value is a short item count (or zero).
>
> The file position indicator for the stream is advanced by the number
> of bytes successfully read or written.
>
> fread() does not distinguish between end-of-file and error, and
> callers must use feof(3) and ferror(3) to determine which occurred.

This means that nread!=0 doesn't make much sense as an end condition for
the loop: nread==0 doesn't necessarily mean that EOF has been reached or
an error has occured (but that is usually the case) and nread!=0 doesn't
necessarily mean that EOF has not been reached or that no read errors
have occured. feof(3) and ferror(3) should be uses when using fread(3).

Currently curl has to performs an extra fread(3) call to get a return
value equal to 0 to stop looping.

This usually "works" (even though nread==0 shouldn't be interpreted as
EOF) if stdin is a pipe because EOF usually marks the "real" end of the
stream, so the extra fread(3) call will return immediately and the extra
read syscall won't be noticeable:

    bash-5.1$ strace -e read curl -s -F file=@- 0x0.st <<< a 2>&1 |
    > tail -n 5
    read(0, "a\n", 4096)                    = 2
    read(0, "", 4096)                       = 0
    read(0, "", 4096)                       = 0
    http://0x0.st/oRs.txt
    +++ exited with 0 +++
    bash-5.1$

But this doesn't work if curl is reading from stdin, stdin is a
terminal, and the EOF is being emulated using a shell with ^D. Two
consecutive ^D will be required in this case to actually make curl stop
reading:

    bash-5.1$ curl -F file=@- 0x0.st
    a
    ^D^D
    http://0x0.st/oRs.txt
    bash-5.1$

A possible workaround to this issue is to use a program that handles EOF
correctly to indirectly send data to curl's stdin:

    bash-5.1$ cat - | curl -F file=@- 0x0.st
    a
    ^D
    http://0x0.st/oRs.txt
    bash-5.1$

This patch makes curl handle EOF properly when using fread(3) in
file2memory() so that the workaround is not necessary.

Since curl was previously ignoring read errors caused by this fread(3),
ferror(3) is also used in the condition of the loop: read errors and EOF
will have the same meaning; this is done to somewhat preserve the old
behaviour instead of making the command fail when a read error occurs.

Closes #8701
2022-04-17 11:36:28 +02:00
Daniel Stenberg
01a1697a79
curl: error out if -T and -d are used for the same URL
As one implies PUT and the other POST, both cannot be used
simultaneously.

Add test 378 to verify.

Reported-by: Boris Verkhovskiy
Fixes #8704
Closes #8715
2022-04-17 11:18:17 +02:00
Daniel Stenberg
36e09dc302
fail.d: tweak the description
Reviewed-by: Daniel Gustafsson
Suggested-by: Robert Charles Muir
Ref: https://twitter.com/rcmuir/status/1514915401574010887

Closes #8714
2022-04-15 23:50:35 +02:00
Daniel Stenberg
4a8f6869db
English: use American spelling consistently
Authorization, Initialization, Organization etc.

Closes #8673
2022-04-05 14:55:47 +02:00
Daniel Stenberg
f48a0ce814
tool_listhelp.c: uppercase URL 2022-04-04 14:43:33 +02:00
Daniel Stenberg
a3f4d7cee9
misc: spelling fixes
Mostly in comments but also in the -w documentation for headers_json.

Closes #8647
2022-03-30 10:49:06 +02:00
Ian Blanes
9e5bd9ba19
curl: fix segmentation fault for empty output file names.
Function glob_match_url set *result to NULL when called with filename =
"", producing an indirect NULL pointer dereference.

Closes #8606
2022-03-28 09:39:09 +02:00
Daniel Stenberg
e7793cb57b
curl/header_json: output the header names in lowercase
To better allow json[“header”].

Reported-by: Peter Korsgaard
Bug: https://daniel.haxx.se/blog/2022/03/24/easier-header-picking-with-curl/comment-page-1/#comment-25878
Closes #8633
2022-03-25 11:24:27 +01:00
Daniel Stenberg
8e22fc68e7
scripts: move three scripts from lib/ to scripts/
Move checksrc.pl, firefox-db2pem.sh and mk-ca-bundle.pl since they don't
particularly belong in lib/

Also created an EXTRA_DIST= in scripts/Makefile.am instead of specifying
those files in the root Makefile.am

Closes #8625
2022-03-23 15:26:11 +01:00
Jay Satiro
cabcf403ed docs/opts: Mention Schannel client cert type is P12
Schannel backend code behaves same as Secure Transport, it expects a P12
certificate file or the name of a certificate already in the user's OS
key store. Also, both backends ignore CURLOPT_SSLKEY (tool: --key)
because they expect the private key to already be available from the
keystore or P12 certificate.

Ref: https://github.com/curl/curl/discussions/8581#discussioncomment-2337260

Closes https://github.com/curl/curl/pull/8587
2022-03-22 14:24:06 -04:00
Daniel Stenberg
4133a69f2d
curl: add %{header_json} support in -w handling
Outputs all response headers as a JSON object.
2022-03-22 08:24:25 +01:00
Daniel Stenberg
2d45137e1e
curl: add %header{name} support in -w handling
Outputs the response header 'name'
2022-03-22 08:24:25 +01:00
Jay Satiro
d3cb3be520 lib: fix some misuse of curlx_convert_wchar_to_UTF8
curlx_convert_wchar_to_UTF8 must be freed by curlx_unicodefree, but
prior to this change some uses mistakenly called free.

I've reviewed all other uses of curlx_convert_wchar_to_UTF8 and
curlx_convert_UTF8_to_wchar.

Ref: https://github.com/curl/curl/commit/1d5d0ae

Closes https://github.com/curl/curl/pull/8521
2022-03-18 03:20:03 -04:00
Marc Hoersken
8b42d7b9d4
tool and tests: force flush of all buffers at end of program
On Windows data can be lost in buffers in case of abnormal program
termination, especially in process chains as seen due to flaky tests.
Therefore flushing all buffers manually should avoid this data loss.

In the curl tool we play the safe game by only flushing write buffers,
but in the testsuite where we manage all buffers, we flush everything.

This should drastically reduce Windows CI and testsuite flakiness.

Reviewed-by: Daniel Stenberg

Supersedes #7833 and #6064
Closes #8516
2022-03-13 13:29:28 +01:00
Dan Fandrich
3055c4c814 tool_cb_hdr: Turn the Location: into a terminal hyperlink
This turns even relative URLs into clickable hyperlinks in a supported
terminal when --styled-output is enabled. Many terminals already turn
URLs into clickable links but there is not enough information in a
relative URL to do this automatically otherwise.
2022-03-11 17:25:35 -08:00
HexTheDragon
1831a6e7f1
curl: add --no-clobber
Does not overwrite output files if they already exist

Closes #7708
Co-authored-by: Daniel Stenberg
2022-03-11 08:38:01 +01:00
Daniel Stenberg
08a96c6e4e
curl: add --remove-on-error
If a transfer returns an error, using this option makes curl remove the
leftover downloded (partial) local file before exiting.

Added test 376 to verify

Closes #8503
2022-03-10 19:57:55 +01:00
Daniel Stenberg
bec62e39d1
tool_operate: fix a scan-build warning
... and avoid the temp storing of the return code in a diff variable.

Closes #8565
2022-03-10 08:31:20 +01:00
Daniel Stenberg
95e8515ca0
curl: error out when options need features not present in libcurl
Trying to use a proxy when libcurl was built with proxy support disabled
should make curl error out properly.

Remove knowledge of disabled features from the tool code and instead
make it properly respond to what libcurl returns. Update all tests to
properly require the necessary features to be present/absent so that the
test suite can still be run even with libcurl builds with disabled
features.

Ref: https://curl.se/mail/archive-2022-03/0013.html
Closes #8565
2022-03-10 08:30:45 +01:00
Daniel Stenberg
2610142139
lib: remove support for CURL_DOES_CONVERSIONS
TPF was the only user and support for that was dropped.

Closes #8378
2022-02-04 08:05:35 +01:00
Daniel Stenberg
da15443ddd
TPF: drop support
There has been no TPF related changes done since September 2010 (commit
7e1a45e224) and since this is a platform that is relatively different
than many others (== needs attention), I draw the conclusion that this
build is broken since a long time.

Closes #8378
2022-02-04 08:05:35 +01:00
Daniel Stenberg
32160cae84
tool_getparam: initial --json support
Adds these test cases:

 383 - simple single command line option
 384 - reading it from stdin
 385 - getting two --json options on command line
 386 - --next works after --json

Closes #8314
2022-02-01 10:39:55 +01:00
Daniel Stenberg
3b16575ae9
netware: remove support
There are no current users and no Netware related changes done in the
code for over 13 years is a clear sign this is abandoned.

Closes #8358
2022-01-31 13:40:26 +01:00
Daniel Stenberg
f6088822ce
misc: remove the final watcom references
Follow-up to bbf8cae44d

We removed support for the watcom builds files back in September
2020. This removes all remaining watcom references and ifdefs.

Closes #8287
2022-01-17 08:49:25 +01:00
Daniel Stenberg
0041fe7ef5
tool_getparam: DNS options that need c-ares now fail without it
Just silently accepting the options and then not having any effect is
not good.

Ref: #8283
Closes #8285
2022-01-17 08:44:41 +01:00
Daniel Stenberg
f974bee623
curl: remove "separators" (when using globbed URLs)
Unless muted (with -s) When doing globbing, curl would output mime-like
separators between the separate transfers. This is not documented
anywhere, surprises users and clobbers the output. Gone now.

Updated test 18 and 1235

Reported-by: jonny112 on github
Bug: https://github.com/curl/curl/discussions/8257
Closes #8278
2022-01-15 23:41:28 +01:00
Patrick Monnerat
b6acbdeb63
curl tool: erase some more sensitive command line arguments
As the ps command may reveal sensitive command line info, obfuscate
options --tlsuser, --tlspasswd, --proxy-tlsuser, --proxy-tlspassword and
--oauth2-bearer arguments.

Reported-by: Stephen Boost <s.booth@epcc.ed.ac.uk>

Closes #7964
2022-01-10 15:49:25 +01:00
Daniel Stenberg
e180dde9c5
tool_findfile: free mem properly
Follow-up to 764e4f066d

Closes #8242
2022-01-07 16:39:19 +01:00
Daniel Stenberg
764e4f066d
tool_findfile: check ~/.config/curlrc too
... after the initial checks for .curlrc and if XDG_CONFIG_HOME is not
set, use $HOME and $CURL_HOME to check if ~/.config/curlrc is present.

Add test 436 to verify

Reported-by: Sandro Jaeckel
Fixes #8208
Closes #8213
2022-01-07 10:10:49 +01:00
Daniel Stenberg
ee2ca5826b
tool_operate: warn if too many output arguments were found
More output instructions than URLs is likely a user error.

Add test case 371 to verify

Closes #8210
2022-01-03 15:55:38 +01:00
Daniel Stenberg
21248e052d
checksrc: detect more kinds of NULL comparisons we avoid
Co-authored-by: Jay Satiro
Closes #8180
2021-12-27 23:39:26 +01:00
Daniel Stenberg
bc064a8177
tool_parsecfg: use correct free() call to free memory
Detected by Coverity. CID 1494642.
Follow-up from 2be1aa619b

Closes #8099
2021-12-06 11:00:30 +01:00
Daniel Stenberg
75b31ce626
tool_operate: fix potential memory-leak
A 'CURLU *' would leak if url_proto() is called with no URL.

Detected by Coverity. CID 1494643.
Follow-up to 18270893ab
Closes #8098
2021-12-06 09:59:38 +01:00
Daniel Stenberg
a82fe5d2d0
tool_listhelp: sync
Follow-up to 172068b76f
2021-12-05 23:14:11 +01:00
Daniel Stenberg
cdde5414fc
writeout: fix %{http_version} for HTTP/3
Output "3" properly when HTTP/3 was used.

Reported-by: Bernat Mut
Fixes #8072
Closes #8092
2021-12-03 23:00:37 +01:00
Wyatt O'Day
7da636cad5 version_win32: Check build number and platform id
Prior to this change the build number was not checked during version
comparison, and the platform id was supposed to be checked but wasn't.

Checking the build number is required for enabling "evergreen"
Windows 10/11 features (like TLS 1.3).

Ref: https://github.com/curl/curl/pull/7784

Closes https://github.com/curl/curl/pull/7824
Closes https://github.com/curl/curl/pull/7867
2021-12-02 03:39:14 -05:00
Daniel Stenberg
2be1aa619b
tool_findfile: search for a file in the homedir
The homedir() function is now renamed into findfile() and iterates over
all the environment variables trying to access the file in question
until it finds it. Last resort is then getpwuid() if
available. Previously it would first try to find a home directory and if
that was set, insist on checking only that directory for the file. This
now returns the full file name it finds.

The Windows specific checks are now done differently too and in this
order:

1 - %USERPROFILE%
2 - %APPDATA%
3 - %USERPROFILE%\\Application Data

The windows order is modified to match how the Windows 10 ssh tool works
when it searches for .ssh/known_hosts.

Reported-by: jeffrson on github
Co-authored-by: Jay Satiro
Fixes #8033
Closes #8035
2021-11-29 00:57:22 +01:00
Viktor Szakats
8c0336cf5d
Makefile.m32: rename -winssl option to -schannel and tidy up
- accept `-schannel` as an alternative to `CFG` option `-winssl`
  (latter still accepted, but deprecated)
- rename internal variable `WINSSL` to `SCHANNEL`
- make the `CFG` option evaluation shorter, without repeating the option
  name

Reviewed-by: Marcel Raad
Reviewed-by: Daniel Stenberg
Closes #8053
2021-11-25 17:36:38 +00:00
Daniel Stenberg
ffb6a9e8a6
curl: improve error message for --head with -J
... it now focuses on the "output of headers" combined with the
--remote-header-name option, as that is actually the problem. Both
--head and --include can output headers.

Reported-by: nimaje on github
Fixes #7987
Closes #8045
2021-11-23 08:43:44 +01:00
Daniel Stenberg
18270893ab
tool_operate: only set SSH related libcurl options for SSH URLs
For example, this avoids trying to find and set the known_hosts file (or
warn for its absence) if SFTP or SCP are not used.

Closes #8040
2021-11-21 23:12:48 +01:00