RFC 7616 (and 2617) requires values to be "unquoted" before used for
digest calculations. The only place where unquoting can be done
correctly is header parsing function (realm="DOMAIN\\host" and
realm=DOMAN\\host are different realms).
This commit adds unquoting (de-escaping) of all values during header
parsing and quoting of the values during header forming. This approach
should be most straightforward and easy to read/maintain as all values
are processed in the same way as required by RFC.
Closes#8912
The generated stdout data is written in binary mode with [LF]
line endings, therefore we also need to do a binary comparison.
Assisted-by: Jay Satiro
Assisted-by: Daniel Stenberg
Follow up to c9b60f0053Fixes#8920Closes#8936
This test is contributing to flakiness on the Windows CI runs.
Killing the ftp server after the test run like other slowness
tests already do may help resolve or reduce the flakiness.
Closes#8907
Folded header lines will now get passed through like before. The headers
API is adapted and will provide the content unfolded.
Added test 1274 and extended test 1940 to verify.
Reported-by: Petr Pisar
Fixes#8844Closes#8899
Commit 709ae2454f added a fake hostname to avoid leaking the local
hostname, but omitted copying it to the host buffer. Fix by copying
and adjust the test fallout.
Closes: #8895Fixes: #8893
Reported-by: Patrick Monnerat <patrick@monnerat.net>
Reviewed-by: Daniel Stenberg <daniel@haxx.se>
Usage:
curl -x "socks5h://localhost/run/tor/socks" "https://example.com"
Updated runtests.pl to run a socksd server listening on unix socket
Added tests test1467 test1468
Added documentation for proxy command line option and socks proxy
options
Closes#8668
These two options were only ever used for the OpenSSL backend for
versions before 1.1.0. They were never used for other backends and they
are not used with recent OpenSSL versions. They were never used much by
applications.
The defines RANDOM_FILE and EGD_SOCKET can still be set at build-time
for ancient EOL OpenSSL versions.
Closes#8670
Commit 46d45ea3a incorrectly hardcoded the User-Agent in the test
output file which breaks when curlver is updated. Shift to using
the %VERSION macro instead.
Closes: #8856
This makes it more likely that the trailer is received
seperately from the last-chunk.
curl doesn't seem to care about this but it makes the tests
more useful when testing external proxies like Privoxy.
This allows to use write delays for large responses without
resulting in the test taking an unreasonable amount of time.
In many cases delaying writes by a whole second or more isn't
necessary for the desired effect.
Closes#8827
The asterisk in the abbreviation *NIX (for UNIX/Linux) needs to be
escaped to not mean start of italic formatting. This is consistent
with docs/RELEASE-PROCEDURE.md.
Closes: #8802
Reviewed-by: Daniel Stenberg <daniel@haxx.se>
Silences the warning:
CC socksd-socksd.o
socksd.c:143:13: warning: no previous extern declaration for
non-static variable 'reqlogfile' [-Wmissing-variable-declarations]
const char *reqlogfile = DEFAULT_REQFILE;
^
socksd.c:143:7: note: declare 'static' if the variable is not
intended to be used outside of this translation unit
const char *reqlogfile = DEFAULT_REQFILE;
^
1 warning generated.
... when compiling with clang 13.
Closes: #8799
Reviewed-by: Daniel Gustafsson <daniel@yesql.se>
Hyper now has the ability to preserve header order. This commit adds a
few lines setting the connection options for this feature.
Related to issue #8617Closes#8707
- Test 973 redirects from HTTP to FTP, clear auth
- Test 974 redirects from HTTP to HTTP different port, clear auth
- Test 975 redirects from HTTP to FTP, permitted to keep auth
- Test 976 redirects from HTTP to HTTP different port, permitted to keep
auth
In order to avoid the risk of it being used in an accidental trigraph in
the generated code.
Reported-by: Harry Sintonen
Bug: https://hackerone.com/reports/1548535Closes#8742
Also move timediff_t definitions from timeval.h to timediff.h and
then make timeval.h include the new standalone-capable timediff.h.
Reviewed-by: Jay Satiro
Reviewed-by: Daniel Stenberg
Supersedes #5888Closes#8595
As one implies PUT and the other POST, both cannot be used
simultaneously.
Add test 378 to verify.
Reported-by: Boris Verkhovskiy
Fixes#8704Closes#8715
This leaves the CURLE_RECV_ERROR error code for explicit failure to
receive network data and allows users to better separate the problems.
Ref #8356
Reported-by: Rianov Viacheslav
Closes#8506
Move checksrc.pl, firefox-db2pem.sh and mk-ca-bundle.pl since they don't
particularly belong in lib/
Also created an EXTRA_DIST= in scripts/Makefile.am instead of specifying
those files in the root Makefile.am
Closes#8625
There is no need to test for both _WIN32 and WIN32 as curl_setup.h
automatically defines the later if the first one is defined.
Also tests/server/util.c is only checking for WIN32 arouund the
implementation of win32_perror, so just defining _WIN32
would not be sufficient for a successful compilation.
Reviewed-by: Daniel Stenberg
Reviewed-by: Jay Satiro
Closes#8594
This test verifies that the order of functions in public headers remain
the same but hasn't been updated to care for recently added header
files. The order is important for some few platforms - or VERSIONINFO
needs to updated.
This fix also updates VERSIONINFO to be sure.
Closes#8620
Make tests require h2c feature present to run, and only set h2c if
nghttp2 is used in the build. Hyper does not support it.
Remove those tests from DISABLED
Fixes#8605Closes#8613
They are not allowed by the protocol and allowing them risk that curl
misbehaves somewhere where C functions are used but won't work on the
full contents. Further, they are not supported by hyper and they cause
problems for the new coming headers API work.
Updated test 262 to verify and enabled it for hyper as well
Closes#8601
On Windows data can be lost in buffers in case of abnormal program
termination, especially in process chains as seen due to flaky tests.
Therefore flushing all buffers manually should avoid this data loss.
In the curl tool we play the safe game by only flushing write buffers,
but in the testsuite where we manage all buffers, we flush everything.
This should drastically reduce Windows CI and testsuite flakiness.
Reviewed-by: Daniel Stenberg
Supersedes #7833 and #6064Closes#8516
If a transfer returns an error, using this option makes curl remove the
leftover downloded (partial) local file before exiting.
Added test 376 to verify
Closes#8503
Trying to use a proxy when libcurl was built with proxy support disabled
should make curl error out properly.
Remove knowledge of disabled features from the tool code and instead
make it properly respond to what libcurl returns. Update all tests to
properly require the necessary features to be present/absent so that the
test suite can still be run even with libcurl builds with disabled
features.
Ref: https://curl.se/mail/archive-2022-03/0013.htmlCloses#8565
The 'oldlibssh' feature indicates that the error code returned by libssh
for a broken known_hosts file should be 67 rather than 60 (test1459).
This feature was added as part of #8444 with 'oldlibssh' mapping to
libssh versions prior to 0.9.6, and then refined as part of #8511 to map
to versions prior to 0.9.5.
In Red Hat Enterprise Linux 8.5 there is a patched version of libssh
version 0.9.4 (https://git.centos.org/rpms/libssh/blob/c8/f/SOURCES) in
which test1459 fails because it returns the "new" value rather than the
"old" one. It's plausible that one of the patches is responsible for
this rather than the underlying code but I don't think so.
This change therefore drops the 'oldlibssh' version check to map to
libssh versions older than 0.9.4, which fixes builds on RHEL-8.
Closes#8548
In March 2010 (commit 4259d2df7d) we removed the embedded 'ares'
directory from the curl source tree but we have since supported
especially detecting and using that build directory. The time has come
to remove that kludge and ask users to specify the c-ares dir correctly
with --enable-ares.
Closes#8397
Test 1165 would fail on some systems because it didn't detect
CURL_DISABLE_* symbols that were used to the right of another one on the
same line! The script would only detect and extract the first one.
Reported-by: Marcel Raad
Fixes#8384Closes#8388
There has been no TPF related changes done since September 2010 (commit
7e1a45e224) and since this is a platform that is relatively different
than many others (== needs attention), I draw the conclusion that this
build is broken since a long time.
Closes#8378
Silences the following warnings when using a Makefile.inc-free
TESTDIR using the "-o" argument:
readline() on closed filehandle D at ./runtests.pl line 592.
Use of uninitialized value $disttests in pattern match (m//) at
./runtests.pl line 3602.
Closes https://github.com/curl/curl/pull/8379
Adds these test cases:
383 - simple single command line option
384 - reading it from stdin
385 - getting two --json options on command line
386 - --next works after --json
Closes#8314
- Disable all MSYS2 path transformation in test3021 and test3022.
Prior to this change path transformation in those tests was disabled
only for arguments that start with forward slashes. However arguments
that are in base64 contain forward slashes at any position and caused
unwanted translations.
== Info: Denied establishing ssh session: mismatch sha256 fingerprint.
Remote +/EYG2YDzDGm6yiwepEMSuExgRRMoTi8Di1UN3kixZw= is not equal to
+C:/msys64/EYG2YDzDGm6yiwepEMSuExgRRMoTi8Di1UN3kixZw
In the above example an argument containing a base64 sha256 fingerprint
was passed to curl after MSYS2 translated +/ into +C:/msys64/, and then
the fingerprint didn't match what was expected.
Ref: https://www.msys2.org/wiki/Porting/
Fixes https://github.com/curl/curl/issues/8084
Closes https://github.com/curl/curl/pull/8325
Reverts 5de8d84098 (May 2014, shipped in 7.37.0) and the
follow-up changes done afterward.
Keep the dot in names for everything except the SNI to make curl behave
more similar to current browsers. This means 'name' and 'name.' send the
same SNI for different 'Host:' headers.
Updated test 1322 accordingly
Fixes#8290
Reported-by: Charles Cazabon
Closes#8320
This is the total number of bytes allocated, increasing for new
allocations and never reduced when freed. The existing "Maximum
allocated" is the high water mark.
Closes#8330
Follow-up to bbf8cae44d
We removed support for the watcom builds files back in September
2020. This removes all remaining watcom references and ifdefs.
Closes#8287
Unless muted (with -s) When doing globbing, curl would output mime-like
separators between the separate transfers. This is not documented
anywhere, surprises users and clobbers the output. Gone now.
Updated test 18 and 1235
Reported-by: jonny112 on github
Bug: https://github.com/curl/curl/discussions/8257Closes#8278
Mesalink has ceased development. We can no longer encourage use of it.
It seems to be continued under the name TabbySSL, but no attempts have
(yet) been to make curl support it.
Fixes#8188Closes#8191
For consistency, use the same return code for URL malformats,
independently of what scheme that is used. Previously this would return
CURLE_LDAP_INVALID_URL, but starting now that error cannot be returned.
Closes#8170
... after the initial checks for .curlrc and if XDG_CONFIG_HOME is not
set, use $HOME and $CURL_HOME to check if ~/.config/curlrc is present.
Add test 436 to verify
Reported-by: Sandro Jaeckel
Fixes#8208Closes#8213
The callbacks were partially documented to support this. Now the
behavior is documented and returning error from either of these
callbacks will effectively kill all currently ongoing transfers.
Added test 530 to verify
Reported-by: Marcelo Juchem
Fixes#8083Closes#8089
Make all libcurl related options use .nf (no fill) for the SYNOPSIS
section - for consistent look. roffit then renders that section using
<pre> (monospace font) in html for the website.
Extended manpage-syntax (test 1173) with a basic check for it.
Closes#8062
Previously, the return code CURLUE_MALFORMED_INPUT was used for almost
30 different URL format violations. This made it hard for users to
understand why a particular URL was not acceptable. Since the API cannot
point out a specific position within the URL for the problem, this now
instead introduces a number of additional and more fine-grained error
codes to allow the API to return more exactly in what "part" or section
of the URL a problem was detected.
Also bug-fixes curl_url_get() with CURLUPART_ZONEID, which previously
returned CURLUE_OK even if no zoneid existed.
Test cases in 1560 have been adjusted and extended. Tests 1538 and 1559
have been updated.
Updated libcurl-errors.3 and curl_url_strerror() accordingly.
Closes#8049
file URLs that are 6 bytes or shorter are not complete. Return
CURLUE_MALFORMED_INPUT for those. Extended test 1560 to verify.
Triggered by #8041Closes#8042
This is done by having native code do the haproxy header output before
hyper issues its request. The little downside with this approach is that
we need the entire Curl_buffer_send() function built, which is otherwise
not used for hyper builds.
If hyper ends up getting native support for the haproxy protocols we can
backpedal on this.
Enables test 1455 and 1456
Closes#8034
Until now, form field and file names where escaped using the
backslash-escaping algorithm defined for multipart mails. This commit
replaces this with the percent-escaping method for URLs.
As this may introduce incompatibilities with server-side applications, a
new libcurl option CURLOPT_MIME_OPTIONS with bitmask
CURLMIMEOPT_FORMESCAPE is introduced to revert to legacy use of
backslash-escaping. This is controlled by new cli tool option
--form-escape.
New tests and documentation are provided for this feature.
Reported by: Ryan Sleevi
Fixes#7789Closes#7805
Adds Schannel variants of SSLpinning tests that include the option
--ssl-revoke-best-effort to ignore certificate revocation check
failures which is required due to our custom test CA certificate.
Disable the original variants if the Schannel backend is enabled.
Also skip all IDN tests which are broken while using an msys shell.
This is a step to simplify test exclusions for Windows and MinGW.
Reviewed-by: Jay Satiro
Reviewed-by: Marcel Raad
Reviewed-by: Daniel Stenberg
Closes#7968
When failing to create the output file for saving an etag, only fail
that particular single transfer and allow others to follow.
In a serial transfer setup, if no transfer at all is done due to them
all being skipped because of this error, curl will output an error
message and return exit code 26.
Added test 369 and 370 to verify.
Reported-by: Earnestly on github
Ref: #7942Closes#7945
This makes it possible to use -u again for local testing,
but removes the flag from CI config files and make targets.
Reviewed-by: Daniel Stenberg
Partially reverts #7841Closes#7921
... which then also includes negative ones as test 1430 uses.
This makes native + hyper backend act identically on this and therefore
test 1430 can now be enabled when building with hyper. Adjust test 1431
as well.
Closes#7909
This is the same order we already enforce among the options' man pages:
consistency is good. Add lots of previously missing examples.
Adjust the manpage-syntax script for this purpose, used in test 1173.
Closes#7904
This is a continuation of commit ec91b5a690 in which
changing this test was missed. There are no other python2 leftovers
now.
Based on a Debian patch originally written by Alessandro Ghedini
<ghedo@debian.org>
Closes#7899
In order to check the actual code better, checksrc now ignores
everything that look like preprocessor instructions. It also means
that code in macros are now longer checked.
Note that some rules then still don't need to be followed when code is
exactly below a cpp instruction.
Removes two checksrc exceptions we needed previously because of
preprocessor lines being checked.
Reported-by: Marcel Raad
Fixes#7863Closes#7864
... to let curl_easy_escape() itself do the strlen. This avoids a (false
positive) Coverity warning and it avoids us having to store the strlen()
return value in an int variable.
Reviewed-by: Daniel Gustafsson
Closes#7862
- Free the allocated http request struct on cleanup.
Prior to this change if sws was built with leak sanitizer it would
report a memory leak error during testing.
Closes https://github.com/curl/curl/pull/7849
It uses revoked.badssl.com which now is expired and therefor this now
permafails. We should not use external sites for tests, this test should
be converted to use our own infra.
Closes#7845
Report ignore tests separately from the actual fails.
Don't exit non-zero if test servers couldn't get killed.
Assisted-by: Jay Satiro
Fixes#7818Closes#7841
The host name is stored decoded and can be encoded when used to extract
the full URL. By default when extracting the URL, the host name will not
be URL encoded to work as similar as possible as before. When not URL
encoding the host name, the '%' character will however still be encoded.
Getting the URL with the CURLU_URLENCODE flag set will percent encode
the host name part.
As a bonus, setting the host name part with curl_url_set() no longer
accepts a name that contains space, CR or LF.
Test 1560 has been extended to verify percent encodings.
Reported-by: Noam Moshe
Reported-by: Sharon Brizinov
Reported-by: Raul Onitza-Klugman
Reported-by: Kirill Efimov
Fixes#7830Closes#7834
Adjust the description position to make an aligned column when doing
help listings, which is more pleasing to the eye.
Suggested-by: Gisle Vanem
Closes#7792
tool_listhelp.c is now a separate file with only the command line --help
output, exactly as generated by gen.pl. This makes it easier to generate
updates according to what's in the docs/cmdline-opts docs.
cd $srcroot/docs/cmdline-opts
./gen.pl listhelp *.d > $srcroot/src/tool_listhelp.c
With a configure build, this also works:
make -C src listhelp
Closes#7787
Triggered before a request is made but after a connection is set up
Changes:
- callback: Update docs and callback for pre-request callback
- Add documentation for CURLOPT_PREREQDATA and CURLOPT_PREREQFUNCTION,
- Add redirect test and callback failure test
- Note that the function may be called multiple times on a redirection
- Disable new 2086 test due to Windows weirdness
Closes#7477
- file://host.name/path/file.txt is a valid UNC path
\\host.name\path\files.txt to a non-local file transformed into URI
(RFC 8089 Appendix E.3)
- UNC paths on other OSs must be smb: URLs
Closes#7366
Add curl_url_strerror() to convert CURLUcode into readable string and
facilitate easier troubleshooting in programs using URL API.
Extend CURLUcode with CURLU_LAST for iteration in unit tests.
Update man pages with a mention of new function.
Update example code and tests with new functionality where it fits.
Closes#7605
The "content" is delivered as "body" by curl, but the envelope continues
after the body and the rest of it should be delivered as header.
The IMAP server can now get 'POSTFETCH' set to include more data to
include after the body and test 897 is done to verify that such "extra"
header data is in fact delivered by curl as header.
Ref: #7284 but fails to reproduce the issue
Closes#7748
When the "reason phrase" in the HTTP status line starts with a digit,
that was treated as the forth response code digit and curl would claim
the response to be non-compliant.
Added test 1466 to verify this case.
Regression brought by 5dc594e44f
Reported-by: Glenn de boer
Fixes#7738Closes#7739
The test should be fine and it works for me repeated when run manually,
but clearly it causes CI failures and it needs more research.
Reported-by: RiderALT on github
Fixes#7725Closes#7732
... by not using options with no argument where an argument is required:
=== Start of file tests/log/ssh_server.log
curl_sshd_config line 6: no argument after keyword "DenyGroups"
curl_sshd_config line 7: no argument after keyword "AllowGroups"
curl_sshd_config line 10: Deprecated option AuthorizedKeysFile2
curl_sshd_config line 29: Deprecated option KeyRegenerationInterval
curl_sshd_config line 39: Deprecated option RhostsRSAAuthentication
curl_sshd_config line 40: Deprecated option RSAAuthentication
curl_sshd_config line 41: Deprecated option ServerKeyBits
curl_sshd_config line 45: Deprecated option UseLogin
curl_sshd_config line 56: no argument after keyword "AcceptEnv"
curl_sshd_config: terminating, 3 bad configuration options
=== End of file tests/log/ssh_server.log
=== Start of file log/sftp_server.log
curl_sftp_config line 33: Unsupported option "rhostsrsaauthentication"
curl_sftp_config line 34: Unsupported option "rsaauthentication"
curl_sftp_config line 52: no argument after keyword "sendenv"
curl_sftp_config: terminating, 1 bad configuration options
Connection closed.
Connection closed
=== End of file log/sftp_server.log
Closes#7724
When setting a blank expire string, meaning unlimited, curl would pass
TIME_T_MAX to getime_r() when creating the output, while on 64 bit
systems such a large value cannot be convetered to a tm struct making
curl to exit the loop with an error instead. It can't be converted
because the year it would represent doesn't fit in the 'int tm_year'
field!
Starting now, unlimited expiry is instead handled differently by using a
human readable expiry date spelled out as "unlimited" instead of trying
to use a distant actual date.
Test 1660 and 1915 have been updated to help verify this change.
Reported-by: Jonathan Cardoso
Fixes#7720Closes#7721
If a server pipelines future responses within the STARTTLS response, the
former are preserved in the pingpong cache across TLS negotiation and
used as responses to the encrypted commands.
This fix detects pipelined STARTTLS responses and rejects them with an
error.
CVE-2021-22947
Bug: https://curl.se/docs/CVE-2021-22947.html
In imap and pop3, check if TLS is required even when capabilities
request has failed.
In ftp, ignore preauthentication (230 status of server greeting) if TLS
is required.
Bug: https://curl.se/docs/CVE-2021-22946.html
CVE-2021-22946
It should not refer to the uagent string that is allocated and created
for the end server http request, as that pointer may be cleared on
subsequent CONNECT requests.
Added test case 1184 to verify.
Reported-by: T200proX7 on github
Fixes#7705Closes#7707
Let's try to actually handle the server unexpectedly alive
case by first making them visible on CI builds as failures.
This is needed to detect issues with killing of the test
servers completely including nested process chains with
multiple PIDs per test server (including bash and perl).
On Windows/cygwin platforms this is especially helpful with
debugging PID mixups due to cygwin using its own PID space.
Reviewed-by: Daniel Stenberg
Closes#7180
If Retry-After: specifies a period that is longer than what fits within
--retry-max-time, then stop retrying immediately.
Added test 366 to verify.
Reported-by: Kari Pahula
Fixes#7675Closes#7676
In every libcurl option man page there are now 8 mandatory sections that
must use the right name in the correct order and test 1173 verifies
this. Only 14 man pages needed adjustments.
The sections and the order is as follows:
- NAME
- SYNOPSIS
- DESCRIPTION
- PROTOCOLS
- EXAMPLE
- AVAILABILITY
- RETURN VALUE
- SEE ALSO
Reviewed-by: Daniel Gustafsson
Closes#7656
Extended manpage-syntax.pl (run by test 1173) to check that every man
page for a libcurl option has an EXAMPLE section that is more than two
lines. Then fixed all errors it found and added examples.
Reviewed-by: Daniel Gustafsson
Closes#7656
Regression. In d6a37c23a3 (7.75.0) we removed the duplicated storage
(connection + easy handle), so this info needs be extracted again even
for re-used connections.
Add test 435 to verify
Reported-by: Max Dymond
Fixes#7660Closes#7662
By making them look less like http headers, the hyper mode "tweak"
doesn't interfere.
Enable test 2002 and 2003 in hyper builds (and 1280 which is unrelated
but should be enabled).
Closes#7658
Make the built-in HTTP parser behave similar to hyper and reject any
HTTP response using more than 3 digits for the response code.
Updated test 1432 accordingly.
Enabled test 1432 in the hyper builds.
Closes#7641
Since this option is also used for FTP, it needs to work to set for
applications even if hyper doesn't support it for HTTP. Verified by test
1137.
Updated docs to specify that the option doesn't work for HTTP when using
the hyper backend.
Closes#7614
1. Call the internal variable portname (like pidname) everywhere.
2. Have a variable wroteportfile (like wrotepidfile) everywhere.
3. Make sure the file is cleaned up on exit (like pidfile).
4. Add parameter --portfile to usage outputs everywhere.
Reviewed-by: Daniel Stenberg
Replaces #7523Closes#7574
- Use our wait_ms() instead of sleep() since Windows doesn't have the
latter.
- Use a separate variable to keep track of whether the pthread_t thread
id is valid.
On Windows pthread_t is not an integer type. pthread offers no macro for
invalid pthread_t thread id, so validity is kept track of separately.
Closes https://github.com/curl/curl/pull/7527
The length of 'long' in a 32-bit system is 32 bits, which cannot be used
to save timestamps after 2038. Most operating systems have extended
time_t to 64 bits.
Remove the castings to long.
Closes#7466
The pid used for server verification is later stored as pid2 in
the hash of running test servers and therefore used for shutdown.
The pid used for shutdown must be the platform-aware (Win32) pid
to avoid leaking test servers while running them using Cygwin/msys.
Reviewed-by: Jay Satiro
Closes#7481
