Commit Graph

1140 Commits

Author SHA1 Message Date
Dan Fandrich
a2b4391a1d test613: stop showing an error on missing output file
This test would show an error message if the output was missing during
the log post-processing step, but the message was not captured by the
test harness and wasn't useful since the normal golden log file
comparison would the problem more clearly.
2023-10-13 20:28:58 -07:00
Dan Fandrich
f64ecb2bc0 test670: increase the test timeout
This should make it more immune to loaded servers.

Ref: #11328
2023-10-09 14:15:29 -07:00
Stefan Eissing
a383d1372f
test1540: improve reliability
- print that bytes have been received on pausing, but not how many

Closes #12069
2023-10-09 14:07:59 +02:00
Stefan Eissing
56d373033d
test2302: improve reliability
- make result print collected write data, unless
  change in meta flags is detected
- will show same result even when data arrives via
  several writecb invocations

Closes #12068
2023-10-09 14:07:08 +02:00
Dan Fandrich
911d37bb2f test1903: actually verify the cookies after the test
The test otherwise could do just about anything (except leak memory in
debug mode) and its bad behaviour wouldn't be detected. Now, check the
resulting cookie file to ensure the cookies are still there.

Closes #12041
2023-10-05 13:13:43 -07:00
Dan Fandrich
930353d0af test1906: set a lower timeout since it's hit on Windows
msys2 builds actually hit the connect timeout in normal operation, so
lower the timeout from 5 minutes to 5 seconds to reduce test time.

Ref: #11328
Closes #12036
2023-10-05 02:05:13 -07:00
Dan Fandrich
2e5ede8f7f test574: add a timeout to the test
This one hangs occasionally, so this will speed up a test run and allow
logs to be seen when it does.

Closes #12025
2023-10-04 12:15:57 -07:00
Dan Fandrich
2bee7aeb34 tests: propagate errors in libtests
Use the test macros to automatically propagate some errors, and check
and log others while running the tests. This can help in debugging
exactly why a test has failed.
2023-10-04 12:15:57 -07:00
Viktor Szakats
3b6d18bbf6
spelling: fix codespell 2.2.6 typos
Closes #12019
2023-10-03 21:37:56 +00:00
Stefan Eissing
8a940fd55c
tests: increase lib571 timeout from 3s to 30s
- 3s is too short for our CI, making this test fail occasionally
- test usually experiences no delay run locally, so 30s wont hurt

Closes #12013
2023-10-03 14:25:06 +02:00
Viktor Szakats
72f0607488
tests: fix compiler warnings
Seen with llvm 17 on Windows x64.

```
.../curl/tests/server/rtspd.c:136:13: warning: no previous extern declaration for non-static variable 'logdir' [-Wmissing-variable-declarations]
  136 | const char *logdir = "log";
      |             ^
.../curl/tests/server/rtspd.c:136:7: note: declare 'static' if the variable is not intended to be used outside of this translation unit
  136 | const char *logdir = "log";
      |       ^
.../curl/tests/server/rtspd.c:137:6: warning: no previous extern declaration for non-static variable 'loglockfile' [-Wmissing-variable-declarations]
  137 | char loglockfile[256];
      |      ^
.../curl/tests/server/rtspd.c:137:1: note: declare 'static' if the variable is not intended to be used outside of this translation unit
  137 | char loglockfile[256];
      | ^
.../curl/tests/server/fake_ntlm.c:43:13: warning: no previous extern declaration for non-static variable 'logdir' [-Wmissing-variable-declarations]
   43 | const char *logdir = "log";
      |             ^
.../curl/tests/server/fake_ntlm.c:43:7: note: declare 'static' if the variable is not intended to be used outside of this translation unit
   43 | const char *logdir = "log";
      |       ^
.../curl/src/tool_doswin.c:350:8: warning: possible misuse of comma operator here [-Wcomma]
  350 |     ++d, ++s;
      |        ^
.../curl/src/tool_doswin.c:350:5: note: cast expression to void to silence warning
  350 |     ++d, ++s;
      |     ^~~
      |     (void)( )
```

```
.../curl/tests/libtest/lib540.c:146:27: warning: result of comparison 'long' > 2147483647 is always false [-Wtautological-type-limit-compare]
  146 |         int itimeout = (L > (long)INT_MAX) ? INT_MAX : (int)L;
      |                         ~ ^ ~~~~~~~~~~~~~
1 warning generated.

.../curl/tests/libtest/libntlmconnect.c:195:31: warning: result of comparison 'long' > 2147483647 is always false [-Wtautological-type-limit-compare]
  195 |       int itimeout = (timeout > (long)INT_MAX) ? INT_MAX : (int)timeout;
      |                       ~~~~~~~ ^ ~~~~~~~~~~~~~
1 warning generated.

.../curl/tests/libtest/lib591.c:117:31: warning: result of comparison 'long' > 2147483647 is always false [-Wtautological-type-limit-compare]
  117 |       int itimeout = (timeout > (long)INT_MAX) ? INT_MAX : (int)timeout;
      |                       ~~~~~~~ ^ ~~~~~~~~~~~~~
1 warning generated.
.../curl/tests/libtest/lib597.c:99:31: warning: result of comparison 'long' > 2147483647 is always false [-Wtautological-type-limit-compare]
   99 |       int itimeout = (timeout > (long)INT_MAX) ? INT_MAX : (int)timeout;
      |                       ~~~~~~~ ^ ~~~~~~~~~~~~~
1 warning generated.
```

Seen on macOS Intel:
```
.../curl/tests/server/sws.c:440:64: warning: field precision should have type 'int', but argument has type 'size_t' (aka 'unsigned long') [-Wformat]
          msnprintf(logbuf, sizeof(logbuf), "Got request: %s %.*s HTTP/%d.%d",
                                                             ~~^~
1 warning generated.
```

Closes #11925
2023-09-24 21:52:54 +00:00
Viktor Szakats
38029101e2
mingw: delete support for legacy mingw.org toolchain
Drop support for "old" / "legacy" / "classic" / "v1" / "mingw32" MinGW:
  https://en.wikipedia.org/wiki/MinGW, https://osdn.net/projects/mingw/
Its homepage used to be http://mingw.org/ [no HTTPS], and broken now.
It supported the x86 CPU only and used a old Windows API header and
implib set, often causing issues. It also misses most modern Windows
features, offering old versions of both binutils and gcc (no llvm/clang
support). It was last updated 2 years ago.

curl now relies on toolchains based on the mingw-w64 project:
https://www.mingw-w64.org/  https://sourceforge.net/projects/mingw-w64/
https://www.msys2.org/  https://github.com/msys2/msys2
https://github.com/mstorsjo/llvm-mingw
(Also available via Linux and macOS package managers.)

Closes #11625
2023-09-23 09:12:57 +00:00
Daniel Stenberg
a878864a48
test3103: CURLOPT_COOKIELIST test 2023-09-19 08:26:14 +02:00
Dan Fandrich
06cdfad49f tests: log the test result code after each libtest
This makes it easier to determine the test status. Also, capitalize
FAILURE and ABORT messages in log lines to make them easier to spot.
2023-09-16 08:33:45 -07:00
Dan Fandrich
c725ec72a3 tests: increase TEST_HANG_TIMEOUT in two tests
These tests had a 5 second timeout compared to 60 seconds for all other
tests. Make these consistent with the others for more reliability on
heavily-loaded machines.

Ref: #11328
2023-09-13 11:26:08 -07:00
Dan Fandrich
f0e4fa445d test661: return from test early in case of curl error 2023-09-13 11:26:08 -07:00
Dan Fandrich
ae84a52c42 test1592: greatly increase the maximum test timeout
It was too short to be reliable on heavily loaded CI machines, and
as a fail-safe only, it didn't need to be short.

Ref: #11328
2023-09-13 11:26:08 -07:00
Dan Fandrich
7d56d2e50d test: minor test cleanups
Remove an obsolete block of code in tests 2032 & 576.
Add a comment in test 1474.
2023-09-13 11:26:08 -07:00
Jay Satiro
b5c65f8b7b http_aws_sigv4: handle no-value user header entries
- Handle user headers in format 'name:' and 'name;' with no value.

The former is used when the user wants to remove an internal libcurl
header and the latter is used when the user actually wants to send a
no-value header in the format 'name:' (note the semi-colon is converted
by libcurl to a colon).

Prior to this change the AWS header import code did not special case
either of those and the generated AWS SignedHeaders would be incorrect.

Reported-by: apparentorder@users.noreply.github.com

Ref: https://curl.se/docs/manpage.html#-H

Fixes https://github.com/curl/curl/issues/11664
Closes https://github.com/curl/curl/pull/11668
2023-09-11 15:24:05 -04:00
Nathan Moinvaziri
f6700c744b schannel: fix ordering of cert chain info
- Use CERT_CONTEXT's pbCertEncoded to determine chain order.

CERT_CONTEXT from SECPKG_ATTR_REMOTE_CERT_CONTEXT contains
end-entity/server certificate in pbCertEncoded. We can use this pointer
to determine the order of certificates when enumerating hCertStore using
CertEnumCertificatesInStore.

This change is to help ensure that the ordering of the certificate chain
requested by the user via CURLINFO_CERTINFO has the same ordering on all
versions of Windows.

Prior to this change Schannel certificate order was reversed in 8986df80
but that was later reverted in f540a39b when it was discovered that
Windows 11 22H2 does the reversal on its own.

Ref: https://github.com/curl/curl/issues/9706

Closes https://github.com/curl/curl/pull/11632
2023-09-08 03:47:13 -04:00
Dave Cottlehuber
a86fcb284f
ws: fix spelling mistakes in examples and tests
Closes #11784
2023-09-03 18:01:58 +02:00
Viktor Szakats
ce3dce9015
tidy-up: mostly whitespace nits
- delete completed TODO from `./CMakeLists.txt`.
- convert a C++ comment to C89 in `./CMake/CurlTests.c`.
- delete duplicate EOLs from EOF.
- add missing EOL at EOF.
- delete whitespace at EOL (except from expected test results).
- convert tabs to spaces.
- convert CRLF EOLs to LF in GHA yaml.
- text casing fixes in `./CMakeLists.txt`.
- fix a codespell typo in `packages/OS400/initscript.sh`.

Closes #11772
2023-08-31 23:02:10 +00:00
Patrick Monnerat
36d656fe57
tests: fix compilation error for os400
OS400 uses BSD 4.3 setsockopt() prototype by default: this does not
define parameter as const, resulting in an error if actual parameter is
const. Remove the const keyword from the actual parameter cast: this
works in all conditions, even if the formal parameter uses it.

Closes #11547
2023-08-30 11:38:46 +02:00
Matthias Gatto
b137634ba3 lib: fix aws-sigv4 having date header twice in some cases
When the user was providing the header X-XXX-Date, the header was
re-added during signature computation, and we had it twice in the
request.

Reported-by: apparentorder@users.noreply.github.com

Signed-off-by: Matthias Gatto <matthias.gatto@outscale.com>

Fixes: https://github.com/curl/curl/issues/11738
Closes: https://github.com/curl/curl/pull/11754
2023-08-30 03:41:29 -04:00
Daniel Stenberg
a06de2b772
libtest: use curl_free() to free libcurl allocated data
In several test programs. These mistakes are not detected or a problem
as long as memdebug.h is included, as that provides the debug wrappers
for all memory functions in the same style libcurl internals do it,
which makes curl_free and free effectively the same call.

Reported-by: Nicholas Nethercote
Closes #11746
2023-08-28 13:24:51 +02:00
Alexander Kanavin
c2212c05aa
tests: update cookie expiry dates to far in the future
This allows testing Y2038 with system time set to after that, so that
actual Y2038 issues can be exposed, and not masked by expiry errors.

Fixes #11576
Closes #11610
2023-08-25 20:57:05 +02:00
Daniel Stenberg
887b998e6e
urlapi: setting a blank URL ("") is not an ok URL
Test it in 1560
Fixes #11714
Reported-by: ad0p on github
Closes #11715
2023-08-23 23:24:16 +02:00
Daniel Stenberg
74b87a8af1
lib: move mimepost data from ->req.p.http to ->state
When the legacy CURLOPT_HTTPPOST option is used, it gets converted into
the modem mimpost struct at first use. This data is (now) kept for the
entire transfer and not only per single HTTP request. This re-enables
rewind in the beginning of the second request instead of in end of the
first, as brought by 1b39731.

The request struct is per-request data only.

Extend test 650 to verify.

Fixes #11680
Reported-by: yushicheng7788 on github
Closes #11682
2023-08-17 23:49:57 +02:00
Daniel Stenberg
c350069f64
urlapi: CURLU_PUNY2IDN - convert from punycode to IDN name
Asssisted-by: Jay Satiro
Closes #11655
2023-08-13 15:34:38 +02:00
Daniel Stenberg
06e2fa2b51
test2306: make it use a persistent connection
+ enable verbose already from the start

Closes #11621
2023-08-08 23:27:24 +02:00
Daniel Stenberg
95ca3c53d3
headers: accept leading whitespaces on first response header
This is a bad header fold but since the popular browsers accept this
violation, so does curl now. Unless built with hyper.

Add test 1473 to verify and adjust test 2306.

Reported-by: junsik on github
Fixes #11605
Closes #11607
2023-08-07 12:45:45 +02:00
Daniel Stenberg
51118ba84c
tests: fix bad printf format flags in test code 2023-08-07 10:46:14 +02:00
Daniel Stenberg
49e2443186
urlapi: make sure zoneid is also duplicated in curl_url_dup
Add several curl_url_dup() tests to the general lib1560 test.

Reported-by: Rutger Broekhoff
Bug: https://curl.se/mail/lib-2023-07/0047.html
Closes #11549
2023-08-01 08:00:28 +02:00
Daniel Stenberg
7c8bae0d9c
nss: remove support for this TLS library
Closes #11459
2023-07-29 23:44:28 +02:00
Viktor Szakats
1199308dbc
cmake: support building static and shared libcurl in one go
This patch adds the ability to build a static and shared libcurl library
in a single build session. It also adds an option to select which one to
use when building the curl executable.

New build options:
- `BUILD_STATIC_LIBS`. Default: `OFF`.
  Enabled automatically if `BUILD_SHARED_LIBS` is `OFF`.
- `BUILD_STATIC_CURL`. Default: `OFF`.
  Requires `BUILD_STATIC_LIBS` enabled.
  Enabled automatically if building static libcurl only.
- `STATIC_LIB_SUFFIX`. Default: empty.
- `IMPORT_LIB_SUFFIX`. Default: `_imp` if implib filename would collide
  with static lib name (typically with MSVC) in Windows builds.
  Otherwise empty.

Also:

- Stop setting the `CURL_STATICLIB` macro via `curl_config.h`, and pass
  it directly to the compiler. This also allows to delete a condition
  from `tests/server/CMakeLists.txt`.

- Complete a TODO by following the logic used in autotools (also for
  `LIBCURL_NO_SHARED`), and set `-DCURL_STATICLIB` in `Cflags:` of
  `libcurl.pc` for _static-only_ curl builds.

- Convert an existing CI test to build both shared and static libcurl.

Closes #11505
2023-07-29 00:40:01 +00:00
Stefan Eissing
29f33b3400 http2: fix crash in handling stream weights
- Delay the priority handling until the stream has been opened.

- Add test2404 to reproduce and verify.

Weights may change "on the run", which is why there are checks in
general egress handling. These must not trigger when the stream has not
been opened yet.

Reported-by: jbgoog@users.noreply.github.com

Fixes https://github.com/curl/curl/issues/11379
Closes https://github.com/curl/curl/pull/11384
2023-06-28 16:32:16 -04:00
Dan Fandrich
2c5c7abb15 libtest: display the times after a test timeout error
This is to help with test failure debugging.

Ref: #11328
Closes #11329
2023-06-19 17:14:27 -07:00
Daniel Stenberg
3c9256c8a0
urlapi: have *set(PATH) prepend a slash if one is missing
Previously the code would just do that for the path when extracting the
full URL, which made a subsequent curl_url_get() of the path to
(unexpectedly) still return it without the leading path.

Amend lib1560 to verify this. Clarify the curl_url_set() docs about it.

Bug: https://curl.se/mail/lib-2023-06/0015.html
Closes #11272
Reported-by: Pedro Henrique
2023-06-08 16:08:45 +02:00
Daniel Stenberg
78886afb50
ws: make the curl_ws_meta() return pointer a const
The returned info is read-only for the user.

Closes #11261
2023-06-07 23:37:21 +02:00
Viktor Szakats
3f8fc25720
cmake: add support for "unity" builds
Aka "jumbo" or "amalgamation" builds. It means to compile all sources
per target as a single C source. This is experimental.

You can enable it by passing `-DCMAKE_UNITY_BUILD=ON` to cmake.
It requires CMake 3.16 or newer.

It makes builds (much) faster, allows for better optimizations and tends
to promote less ambiguous code.

Also add a new AppVeyor CI job and convert an existing one to use
"unity" mode (one MSVC, one MinGW), and enable it for one macOS CI job.

Fix related issues:
- add missing include guard to `easy_lock.h`.
- rename static variables and functions (and a macro) with names reused
  across sources, or shadowed by local variables.
- add an `#undef` after use.
- add a missing `#undef` before use.
- move internal definitions from `ftp.h` to `ftp.c`.
- `curl_memory.h` fixes to make it work when included repeatedly.
- stop building/linking curlx bits twice for a static-mode curl tool.
  These caused doubly defined symbols in unity builds.
- silence missing extern declarations compiler warning for ` _CRT_glob`.
- fix extern declarations for `tool_freq` and `tool_isVistaOrGreater`.
- fix colliding static symbols in debug mode: `debugtime()` and
  `statename`.
- rename `ssl_backend_data` structure to unique names for each
  TLS-backend, along with the `ssl_connect_data` struct member
  referencing them. This required adding casts for each access.
- add workaround for missing `[P]UNICODE_STRING` types in certain Windows
  builds when compiling `lib/ldap.c`. To support "unity" builds, we had
  to enable `SCHANNEL_USE_BLACKLISTS` for Schannel (a Windows
  `schannel.h` option) _globally_. This caused an indirect inclusion of
  Windows `schannel.h` from `ldap.c` via `winldap.h` to have it enabled
  as well. This requires `[P]UNICODE_STRING` types, which is apperantly
  not defined automatically (as seen with both MSVS and mingw-w64).
  This patch includes `<subauth.h>` to fix it.
  Ref: https://github.com/curl/curl/runs/13987772013
  Ref: https://dev.azure.com/daniel0244/curl/_build/results?buildId=15827&view=logs&jobId=2c9f582d-e278-56b6-4354-f38a4d851906&j=2c9f582d-e278-56b6-4354-f38a4d851906&t=90509b00-34fa-5a81-35d7-5ed9569d331c
- tweak unity builds to compile `lib/memdebug.c` separately in memory
  trace builds to avoid PP confusion.
- force-disable unity for test programs.
- do not compile and link libcurl sources to libtests _twice_ when libcurl
  is built in static mode.

KNOWN ISSUES:
- running tests with unity builds may fail in cases.
- some build configurations/env may not compile in unity mode. E.g.:
  https://ci.appveyor.com/project/curlorg/curl/builds/47230972/job/51wfesgnfuauwl8q#L250

Ref: https://github.com/libssh2/libssh2/issues/1034
Ref: https://cmake.org/cmake/help/latest/prop_tgt/UNITY_BUILD.html
Ref: https://en.wikipedia.org/wiki/Unity_build

Closes #11095
2023-06-07 13:06:08 +00:00
Daniel Stenberg
ba669d072d
urlapi: scheme starts with alpha
Add multiple tests to lib1560 to verify

Fixes #11249
Reported-by: ad0p on github
Closes #11250
2023-06-05 16:28:27 +02:00
Daniel Stenberg
329889f1ea
lib1560: verify more scheme guessing
- on 2nd level domains
- on names without dots

As mentioned in #11161, "imap.com" will be guessed IMAP

Closes #11219
2023-05-29 23:44:42 +02:00
Daniel Stenberg
6375a65433
urlapi: remove superfluous host name check
... as it is checked later more proper.

Closes #11195
2023-05-25 08:30:20 +02:00
Jay Satiro
6f93d5f604
lib: fix conversion warnings with gcc on macOS 2023-05-21 14:02:31 +02:00
Emanuele Torre
eef076baa6
Revert "urlapi: respect CURLU_ALLOW_SPACE and CURLU_NO_AUTHORITY for redirects"
This reverts commit df6c2f7b54.
(It only keep the test case that checks redirection to an absolute URL
without hostname and CURLU_NO_AUTHORITY).

I originally wanted to make CURLU_ALLOW_SPACE accept spaces in the
hostname only because I thought
curl_url_set(CURLUPART_URL, CURLU_ALLOW_SPACE) was already accepting
them, and they were only not being accepted in the hostname when
curl_url_set(CURLUPART_URL) was used for a redirection.

That is not actually the case, urlapi never accepted hostnames with
spaces, and a hostname with a space in it never makes sense.
I probably misread the output of my original test when I they were
normally accepted when using CURLU_ALLOW_SPACE, and not redirecting.

Some other URL parsers seems to allow space in the host part of the URL,
e.g. both python3's urllib.parse module, and Chromium's javascript URL
object allow spaces (chromium percent escapes the spaces with %20),
(they also both ignore TABs, and other whitespace characters), but those
URLs with spaces in the hostname are useless, neither python3's requests
module nor Chromium's window.location can actually use them.

There is no reason to add support for URLs with spaces in the host,
since it was not a inconsistency bug; let's revert that patch before it
makes it into release. Sorry about that.

I also reverted the extra check for CURLU_NO_AUTHORITY since that does
not seem to be necessary, CURLU_NO_AUTHORITY already worked for
redirects.

Closes #11169
2023-05-21 13:59:04 +02:00
Daniel Stenberg
92772e6d39
urlapi: allow numerical parts in the host name
It can only be an IPv4 address if all parts are all digits and no more than
four parts, otherwise it is a host name. Even slightly wrong IPv4 will now be
passed through as a host name.

Regression from 17a15d8846 shipped in 8.1.0

Extended test 1560 accordingly.

Reported-by: Pavel Kalyugin
Fixes #11129
Closes #11131
2023-05-19 16:01:26 +02:00
Stefan Eissing
408eb87bb3
cf-socket: completely remove the disabled USE_RECV_BEFORE_SEND_WORKAROUND
Closes #11118
2023-05-18 20:55:16 +02:00
Emanuele Torre
df6c2f7b54
urlapi: respect CURLU_ALLOW_SPACE and CURLU_NO_AUTHORITY for redirects
curl_url_set(uh, CURLUPART_URL, redirurl, flags)  was not respecing
CURLU_ALLOW_SPACE and CURLU_NO_AUTHORITY in the host part of redirurl
when redirecting to an absolute URL.

Closes #11136
2023-05-18 20:52:59 +02:00
Emanuele Torre
f198d33e8d
checksrc: disallow spaces before labels
Out of 415 labels throughout the code base, 86 of those labels were
not at the start of the line. Which means labels always at the start of
the line is the favoured style overall with 329 instances.

Out of the 86 labels not at the start of the line:
* 75 were indented with the same indentation level of the following line
* 8 were indented with exactly one space
* 2 were indented with one fewer indentation level then the following
  line
* 1 was indented with the indentation level of the following line minus
  three space (probably unintentional)

Co-Authored-By: Viktor Szakats

Closes #11134
2023-05-18 20:45:04 +02:00
Daniel Stenberg
efa6c644aa
test2306: verify getting a second response with folded headers
Reproduces the isue #11101 and verifies the fix.

Verifies a17b2a503f
2023-05-12 17:50:33 +02:00
Daniel Stenberg
9ce7eee070
checksrc: find bad indentation in conditions without open brace
If the previous line starts with if/while/for AND ends with a closed
parenthesis and there's an equal number of open and closed parentheses
on that line, verify that this line is indented $indent more steps, if
not a cpp line.

Also adjust the fall-out from this fix.

Closes #11054
2023-04-28 23:11:00 +02:00
Stefan Eissing
930c00c259
Websocket en-/decoding
- state is fully kept at connection, since curl_ws_send() and
  curl_ws_rec() have lifetime beyond usual transfers
- no more limit on frame sizes

Reported-by: simplerobot on github
Fixes #10962
Closes #10999
2023-04-25 23:16:51 +02:00
Daniel Stenberg
17c71df421
tests/libtest/lib1900.c: remove
This file was left behind when the rest of the test was previously removed.

Follow-up to e50a877df7
2023-04-17 12:45:14 +02:00
Daniel Stenberg
4cfa5bcc9a
urlapi: cleanups
- move host checks together
- simplify the scheme parser loop and the end of host name parser
- avoid itermediate buffer storing in multiple places
- reduce scope for several variables
- skip the Curl_dyn_tail() call for speed
- detect IPv6 earlier and skip extra checks for such hosts
- normalize directly in dynbuf instead of itermediate buffer
- split out the IPv6 parser into its own funciton
- call the IPv6 parser directly for ipv6 addresses
- remove (unused) special treatment of % in host names
- junkscan() once in the beginning instead of scattered
- make junkscan return error code
- remove unused query management from dedotdotify()
- make Curl_parse_login_details use memchr
- more use of memchr() instead of strchr() and less strlen() calls
- make junkscan check and return the URL length

An optimized build runs one of my benchmark URL parsing programs ~41%
faster using this branch. (compared against the shipped 7.88.1 library
in Debian)

Closes #10935
2023-04-13 08:41:40 +02:00
Daniel Stenberg
309a517ffd
lib1560: verify that more bad host names are rejected
when setting the hostname component of a URL

Closes #10922
2023-04-11 11:33:07 +02:00
Dan Fandrich
6ed0629901 tests: stop using strndup(), which isn't portable
It's not available on Solaris 10, for example. Since this is just test
code that doesn't need to use an optimized system version, replace it
with the implementation copied from tool_cb_hdr.c.
2023-04-10 10:51:07 -07:00
Daniel Stenberg
826e8011d5
urlapi: prevent setting invalid schemes with *url_set()
A typical mistake would be to try to set "https://" - including the
separator - this is now rejected as that would then lead to
url_get(... URL...) would get an invalid URL extracted.

Extended test 1560 to verify.

Closes #10911
2023-04-09 23:23:54 +02:00
Daniel Stenberg
5f307986a3
lib: remove CURLX_NO_MEMORY_CALLBACKS
The only user of this define was 'chkdecimalpoint' - a special purpose
test tool that was built but not used anymore (since 17c18fbc3 - Apr
2020).

Closes #10908
2023-04-08 17:47:10 +02:00
Daniel Stenberg
17a15d8846
urlapi: detect and error on illegal IPv4 addresses
Using bad numbers in an IPv4 numerical address now returns
CURLUE_BAD_HOSTNAME.

I noticed while working on trurl and it was originally reported here:
https://github.com/curl/trurl/issues/78

Updated test 1560 accordingly.

Closes #10894
2023-04-06 09:02:00 +02:00
Daniel Stenberg
f042e1e75d
urlapi: URL encoding for the URL missed the fragment
Meaning that it would wrongly still store the fragment using spaces
instead of %20 if allowing space while also asking for URL encoding.

Discovered when playing with trurl.

Added test to lib1560 to verify the fix.

Closes #10887
2023-04-05 08:30:12 +02:00
Dan Fandrich
70d2fca2f6 tests: move server config files under the pid dir
These files are generated by the test servers and must therefore be
found in the log directory to make them available to only those servers
once multiple test runners are executing in parallel. They must also not
be deleted with the log files, so they are stored in the pidfile
directory.

Ref: #10818
Closes #10875
2023-03-31 23:08:00 -07:00
Dan Fandrich
2dd471d577 tests: limit return code of unit tests and lib tests
Values greater than 125 have special meanings, so cap it there.  Unit
tests and lib tests use the number of failures as the return code, so a
large number of failures (such as test 2601 as a torture test) can
exceed this causing the test to be erroneously reported as having
failed.

Ref: #10720
2023-03-30 19:16:40 -07:00
Dan Fandrich
1fe081ab42 tests: fix lib tests to run with a dynamic log directory
Ref: #10818
2023-03-30 09:53:57 -07:00
dengjfzh
6c6306f300
rtsp: skip malformed RTSP interleaved frame data
Some IP cameras send malformed RTSP interleaved frames sometimes, which
can cause curl_easy_perform return 1 (CURLE_UNSUPPORTED_PROTOCOL).  This
change attempts to skip clearly incorrect RTSP interleaving frame data.

Closes #10808
2023-03-30 09:59:09 +02:00
Daniel Stenberg
f24014c3eb
lib2305: deal with CURLE_AGAIN
The test does a slightly ugly busy-loop for this case but should be
managable due to it likely being a very short moment.

Mention CURLE_AGAIN in curl_ws_recv.3

Fixes #10760
Reported-by: Jay Satiro
Closes #10781
2023-03-16 23:29:12 +01:00
Daniel Stenberg
5ec15ea48f
lib643: LIB644 is never defined, this is dead code
Closes #10765
2023-03-15 00:08:09 +01:00
Daniel Stenberg
7ce1a6ae75
libtest/Makefile.inc: remove superfluous variables
Rely on the defaults when possible.

Closes #10765
2023-03-15 00:08:01 +01:00
Casey Bodley
495d09810a
aws_sigv4: fall back to UNSIGNED-PAYLOAD for sign_as_s3
all s3 requests default to UNSIGNED-PAYLOAD and add the required
x-amz-content-sha256 header. this allows CURLAUTH_AWS_SIGV4 to correctly
sign s3 requests to amazon with no additional configuration

Signed-off-by: Casey Bodley <cbodley@redhat.com>

Closes #9995
2023-03-14 17:04:47 +01:00
Dan Fandrich
0cbfe32fb1 tests: make first.c the same for both lib tests and unit tests
The only difference used to be global variable used in unittest tests.
After cb7ed5a removed individual flag overrides for the unittests, first.c
was no longer recompiled for unit tests to include the flag, so whether it
worked or gave a link error depended on whether it was compiled in
libtest or unittest first. This way also speeds up the build by
eliminating 40 identical compile invocations.

Fixes #10749
2023-03-12 19:14:59 -07:00
Daniel Stenberg
674a0662cf test1903: test use of COOKIEFILE - reset - COOKIEFILE
This also tests for the memory leak bug fixed by parent commit b559ef6f.

Ref: #10694

Closes https://github.com/curl/curl/pull/10712
2023-03-10 03:56:59 -05:00
Daniel Stenberg
0a0c9b6dfa
urlapi: '%' is illegal in host names
Update test 1560 to verify

Ref: #10708
Closes #10711
2023-03-08 15:33:43 +01:00
Daniel Stenberg
54605666ed
lib1560: fix enumerated type mixed with another type
Follow-up to c84c0f9aa3

Closes #10684
2023-03-06 08:14:42 +01:00
Daniel Stenberg
c84c0f9aa3
lib1560: test parsing URLs with ridiculously large fields
In the order of 120K.

Closes #10665
2023-03-03 23:23:53 +01:00
Daniel Stenberg
e4dfe6fc2f
test1960: verify CURL_SOCKOPT_ALREADY_CONNECTED
When returned from the CURLOPT_SOCKOPTFUNCTION, like when we have a
custom socket connected in the app, passed in to libcurl.

Verifies the fix in #10648

Closes #10651
2023-03-03 08:36:25 +01:00
Daniel Stenberg
835682661c
misc: remove support for curl_off_t < 8 bytes
Closes #10597
2023-02-24 17:05:33 +01:00
Daniel Stenberg
bb11969838
lib1560: add a test using %25 in the userinfo in a URL
Closes #10578
2023-02-21 16:10:13 +01:00
Daniel Stenberg
f074602072
lib517: verify time stamps without leading zeroes plus some more 2023-02-20 23:48:25 +01:00
Daniel Stenberg
b30b0c3840
lib1560: add IPv6 canonicalization tests
Closes #10552
2023-02-17 23:22:05 +01:00
Daniel Stenberg
8b27799f8c
urlapi: do the port number extraction without using sscanf()
- sscanf() is rather complex and slow, strchr() much simpler

- the port number function does not need to fully verify the IPv6 address
  anyway as it is done later in the hostname_check() function and doing
  it twice is unnecessary.

Closes #10541
2023-02-17 16:21:26 +01:00
Daniel Stenberg
36fca647a4
libtest: build lib2305 with multibyte as well
Fixes a build regression.

Follow-up to 5a9a04d556
Reported-by: Viktor Szakats
Ref: https://github.com/curl/curl/pull/10475#issuecomment-1426831800

Closes #10477
2023-02-12 11:20:34 +01:00
Daniel Stenberg
5a9a04d556
test2305: send 3 frames, 4097 bytes each, as one message
Receive them using a 256 bytes buffer in a loop.
2023-02-10 08:28:59 +01:00
Jay Satiro
b4b6e4f1fa curl_setup: Disable by default recv-before-send in Windows
Prior to this change a workaround for Windows to recv before every send
was enabled by default. The way it works is a recv is called before
every send and saves the received data, in case send fails because in
Windows apparently that can wipe out the socket's internal received
data buffer.

This feature has led to several bugs because the way libcurl operates
it waits on a socket to read or to write, and may not at all times
check for buffered receive data.

Two recent significant bugs this workaround caused:
- Broken Schannel TLS 1.3 connections (#9431)
- HTTP/2 arbitrary hangs (#10253)

The actual code remains though it is disabled by default. Though future
changes to connection filter buffering could improve the situation IMO
it's just not tenable to manage this workaround.

Ref: https://github.com/curl/curl/issues/657
Ref: https://github.com/curl/curl/pull/668
Ref: https://github.com/curl/curl/pull/720

Ref: https://github.com/curl/curl/issues/9431
Ref: https://github.com/curl/curl/issues/10253

Closes https://github.com/curl/curl/pull/10409
2023-02-09 01:30:10 -05:00
John Bampton
0065b146eb
misc: fix spelling
Closes #10437
2023-02-08 08:30:05 +01:00
Gisle Vanem
62097a7ea4 libtest: add a sleep macro for Windows
.. because sleep() is used in some libtests.

Closes https://github.com/curl/curl/pull/10295
2023-02-05 03:15:07 -05:00
Stefan Eissing
671158242d
connections: introduce http/3 happy eyeballs
New cfilter HTTP-CONNECT for h3/h2/http1.1 eyeballing.
- filter is installed when `--http3` in the tool is used (or
  the equivalent CURLOPT_ done in the library)
- starts a QUIC/HTTP/3 connect right away. Should that not
  succeed after 100ms (subject to change), a parallel attempt
  is started for HTTP/2 and HTTP/1.1 via TCP
- both attempts are subject to IPv6/IPv4 eyeballing, same
  as happens for other connections
- tie timeout to the ip-version HAPPY_EYEBALLS_TIMEOUT
- use a `soft` timeout at half the value. When the soft timeout
  expires, the HTTPS-CONNECT filter checks if the QUIC filter
  has received any data from the server. If not, it will start
  the HTTP/2 attempt.

HTTP/3(ngtcp2) improvements.
- setting call_data in all cfilter calls similar to http/2 and vtls filters
  for use in callback where no stream data is available.
- returning CURLE_PARTIAL_FILE for prematurely terminated transfers
- enabling pytest test_05 for h3
- shifting functionality to "connect" UDP sockets from ngtcp2
  implementation into the udp socket cfilter. Because unconnected
  UDP sockets are weird. For example they error when adding to a
  pollset.

HTTP/3(quiche) improvements.
- fixed upload bug in quiche implementation, now passes 251 and pytest
- error codes on stream RESET
- improved debug logs
- handling of DRAIN during connect
- limiting pending event queue

HTTP/2 cfilter improvements.
- use LOG_CF macros for dynamic logging in debug build
- fix CURLcode on RST streams to be CURLE_PARTIAL_FILE
- enable pytest test_05 for h2
- fix upload pytests and improve parallel transfer performance.

GOAWAY handling for ngtcp2/quiche
- during connect, when the remote server refuses to accept new connections
  and closes immediately (so the local conn goes into DRAIN phase), the
  connection is torn down and a another attempt is made after a short grace
  period.
  This is the behaviour observed with nghttpx when we tell it to  shut
  down gracefully. Tested in pytest test_03_02.

TLS improvements
- ALPN selection for SSL/SSL-PROXY filters in one vtls set of functions, replaces
  copy of logic in all tls backends.
- standardized the infof logging of offered ALPNs
- ALPN negotiated: have common function for all backends that sets alpn proprty
  and connection related things based on the negotiated protocol (or lack thereof).

- new tests/tests-httpd/scorecard.py for testing h3/h2 protocol implementation.
  Invoke:
    python3 tests/tests-httpd/scorecard.py --help
  for usage.

Improvements on gathering connect statistics and socket access.
- new CF_CTRL_CONN_REPORT_STATS cfilter control for having cfilters
  report connection statistics. This is triggered when the connection
  has completely connected.
- new void Curl_pgrsTimeWas(..) method to report a timer update with
  a timestamp of when it happend. This allows for updating timers
  "later", e.g. a connect statistic after full connectivity has been
  reached.
- in case of HTTP eyeballing, the previous changes will update
  statistics only from the filter chain that "won" the eyeballing.
- new cfilter query CF_QUERY_SOCKET for retrieving the socket used
  by a filter chain.
  Added methods Curl_conn_cf_get_socket() and Curl_conn_get_socket()
  for convenient use of this query.
- Change VTLS backend to query their sub-filters for the socket when
  checks during the handshake are made.

HTTP/3 documentation on how https eyeballing works.

TLS improvements
- ALPN selection for SSL/SSL-PROXY filters in one vtls set of functions, replaces
  copy of logic in all tls backends.
- standardized the infof logging of offered ALPNs
- ALPN negotiated: have common function for all backends that sets alpn proprty
  and connection related things based on the negotiated protocol (or lack thereof).

Scorecard with Caddy.
- configure can be run with `--with-test-caddy=path` to specify which caddy to use for testing
- tests/tests-httpd/scorecard.py now measures download speeds with caddy

pytest improvements
- adding Makfile to clean gen dir
- adding nghttpx rundir creation on start
- checking httpd version 2.4.55 for test_05 cases where it is needed. Skipping with message if too old.
- catch exception when checking for caddy existance on system.

Closes #10349
2023-02-02 09:57:34 +01:00
Daniel Stenberg
2bc1d775f5
copyright: update all copyright lines and remove year ranges
- they are mostly pointless in all major jurisdictions
- many big corporations and projects already don't use them
- saves us from pointless churn
- git keeps history for us
- the year range is kept in COPYING

checksrc is updated to allow non-year using copyright statements

Closes #10205
2023-01-03 09:19:21 +01:00
Marcel Raad
49f39dfac9
lib670: make test.h the first include
As in all other lib tests. This avoids a macro redefinition warning for
`_FILE_OFFSET_BITS` visible in the autobuilds.

Closes https://github.com/curl/curl/pull/10182
2022-12-30 21:57:35 +01:00
Stefan Eissing
71b7e01610
lib: connect/h2/h3 refactor
Refactoring of connection setup and happy eyeballing. Move
nghttp2. ngtcp2, quiche and msh3 into connection filters.

 - eyeballing cfilter that uses sub-filters for performing parallel connects
 - socket cfilter for all transport types, including QUIC
 - QUIC implementations in cfilter, can now participate in eyeballing
 - connection setup is more dynamic in order to adapt to what filter did
   really connect.  Relevant to see if a SSL filter needs to be added or
   if SSL has already been provided
 - HTTP/3 test cases similar to HTTP/2
 - multiuse of parallel transfers for HTTP/3, tested for ngtcp2 and quiche

 - Fix for data attach/detach in VTLS filters that could lead to crashes
   during parallel transfers.
 - Eliminating setup() methods in cfilters, no longer needed.
 - Improving Curl_conn_is_alive() to replace Curl_connalive() and
   integrated ssl alive checks into cfilter.
 - Adding CF_CNTRL_CONN_INFO_UPDATE to tell filters to update
   connection into and persist it at the easy handle.

 - Several more cfilter related cleanups and moves:
   - stream_weigth and dependency info is now wrapped in struct
     Curl_data_priority
   - Curl_data_priority members depend is available in HTTP2|HTTP3
   - Curl_data_priority members depend on NGHTTP2 support
   - handling init/reset/cleanup of priority part of url.c
   - data->state.priority same struct, but shallow copy for compares only

 - PROTOPT_STREAM has been removed
   - Curl_conn_is_mulitplex() now available to check on capability

 - Adding query method to connection filters.
   - ngtcp2+quiche: implementing query for max concurrent transfers.

 - Adding is_alive and keep_alive cfilter methods. Adding DATA_SETUP event.
   - setting keepalive timestamp on connect
   - DATA_SETUP is called after the connection has been completely
     setup (but may not connected yet) to allow filters to initialize
     data members they use.

 - there is no socket to be had with msh3, it is unclear how select
   shall work

 - manual test via "curl --http3 https://curl.se" fail with "empty
   reply from server".

 - Various socket/conn related cleanups:
   - Curl_socket is now Curl_socket_open and in cf-socket.c
   - Curl_closesocket is now Curl_socket_close and in cf-socket.c
   - Curl_ssl_use has been replaced with Cur_conn_is_ssl
   - Curl_conn_tcp_accepted_set has been split into
     Curl_conn_tcp_listen_set and Curl_conn_tcp_accepted_set
     with a clearer purpose

Closes #10141
2022-12-30 16:43:19 +01:00
Daniel Stenberg
901392cbb7
urlapi: add CURLU_PUNYCODE
Allows curl_url_get() get the punycode version of host names for the
host name and URL parts.

Extend test 1560 to verify.

Closes #10109
2022-12-26 23:29:23 +01:00
Stefan Eissing
c7fb341c0e
tests: add 3 new HTTP/2 test cases, plus https: support for nghttpx
- a simple https get
- a simple https post
- a multi get of 4 requests and check that same connection was used

Closes #10114
2022-12-23 08:56:53 +01:00
Daniel Stenberg
b151faa083
lib1560: add some basic IDN host name tests
Closes #10094
2022-12-15 22:57:08 +01:00
Daniel Stenberg
c20b35ddae
urlapi: reject more bad letters from the host name: &+()
Follow-up from eb0167ff7d

Extend test 1560 to verify

Closes #10096
2022-12-15 08:23:48 +01:00
Daniel Stenberg
86b4b66c54
test2304: verify websocket handling when connection is closed 2022-12-13 15:13:09 +01:00
Casey Bodley
fcf8f68546
test/aws_sigv4: test cases for content-sha256
1956 adds the sha256 value corresponding to an empty buffer
1957 adds an arbitrary value and confirms that the signature differs from 1956
1958 adds whitespace to 1957 and confirms that the signature matches 1957
1959 adds a value longer than 'char sha_hex[65]' in Curl_output_aws_sigv4()

Signed-off-by: Casey Bodley <cbodley@redhat.com>

Closes #9804
2022-11-25 09:24:43 +01:00
Daniel Stenberg
cde8f28489
test1662: verify formpost, 301 redirect, no rewind possible
Reproduces #9735 and verifies the subsequent fix. The original issue
uses a pipe that cannot be rewound, but this test case instead sets a
callback without rewind ability to get roughly the same properties but
being a much more portable test.
2022-11-25 08:30:29 +01:00
Jay Satiro
856b133f5d test3026: reduce runtime in legacy mingw builds
- Load Windows system libraries secur32 and iphlpapi beforehand, so
  that libcurl's repeated global init/cleanup only increases/decreases
  the library's refcount rather than causing it to load/unload.

Assisted-by: Marc Hoersken

Closes https://github.com/curl/curl/pull/9412
2022-11-18 02:54:57 -05:00
Patrick Monnerat
6967571bf2
lib: feature deprecation warnings in gcc >= 4.3
Add a deprecated attribute to functions and enum values that should not
be used anymore.
This uses a gcc 4.3 dialect, thus is only available for this version of
gcc and newer. Note that the _Pragma() keyword is introduced by C99, but
is available as part of the gcc dialect even when compiling in C89 mode.

It is still possible to disable deprecation at a calling module compile
time by defining CURL_DISABLE_DEPRECATION.

Gcc type checking macros are made aware of possible deprecations.

Some testing support Perl programs are adapted to the extended
declaration syntax.

Several test and unit test C programs intentionally use deprecated
functions/options and are annotated to not generate a warning.

New test 1222 checks the deprecation status in doc and header files.

Closes #9667
2022-11-15 10:57:29 +01:00
Daniel Stenberg
fa0b922761
http: mark it 'this_is_a_follow' in the Location: logic
To make regular auth "reloads" to not count as redirects.

Verified by test 3101

Fixes #9885
Closes #9887
2022-11-12 00:12:46 +01:00
Lorenzo Miniero
0baca08dc9
test3100: RTSP Basic authentication
Closes #9449
2022-11-09 09:40:07 +01:00
Rose
f151ec6c10 lib: fix some type mismatches and remove unneeded typecasts
Many of these castings are unneeded if we change the variables to work
better with each other.

Ref: https://github.com/curl/curl/pull/9823

Closes https://github.com/curl/curl/pull/9835
2022-11-08 03:11:01 -05:00