Commit Graph

5902 Commits

Author SHA1 Message Date
Stefan Eissing
c8096668ae
multi: fix multi_wait() timeout handling
- determine the actual poll timeout *after* all sockets
  have been collected. Protocols and connection filters may
  install new timeouts during collection.
- add debug logging to test1533 where the mistake was noticed

Reported-by: Matt Jolly
Fixes #13782
Closes #13825
2024-05-30 08:29:00 +02:00
Jan Venekamp
2b52fe4115
sectransp: use common code for cipher suite lookup
Take advantage of the Curl_cipher_suite_walk_str() and
Curl_cipher_suite_get_str() functions introduced in commit fba9afe.

Closes #13521
2024-05-29 13:08:14 +02:00
Matthias Gatto
768909d89c
aws-sigv4: url encode the canonical path
Refactors canon_query, so it could use the encoding part of the function
to use it in the path.

As the path doesn't encode '/', but encode '=', I had to add some
conditions to know If I was doing the query or path encoding.

Also, instead of adding a `bool in_path` variable, I use `bool
*found_equals` to know if the function was called for the query or path,
as found_equals is used only in query_encoding.

Test 472 verifies.

Reported-by: Alexander Shtuchkin
Fixes #13754
Closes #13814

Signed-off-by: Matthias Gatto <matthias.gatto@outscale.com>
2024-05-29 13:02:52 +02:00
Daniel Stenberg
4f558d288c
tests: run with "--trace-config all" to provide even more info
in case of problems.

Closes #13791
2024-05-28 09:55:36 +02:00
Viktor Szakats
59dc9f7e69
build: untangle CURLDEBUG and DEBUGBUILD macros
`CURLDEBUG` is meant to enable memory tracking, but in a bunch of cases,
it was protecting debug features that were supposed to be guarded with
`DEBUGBUILD`.

Replace these uses with `DEBUGBUILD`.

This leaves `CURLDEBUG` uses solely for its intended  purpose: to enable
the memory tracking debug feature.

Also:
- autotools: rely on `DEBUGBUILD` to enable `checksrc`.
  Instead of `CURLDEBUG`, which worked in most cases because debug
  builds enable `CURLDEBUG` by default, but it's not accurate.
- include `lib/easyif.h` instead of keeping a copy of a declaration.
- add CI test jobs for the build issues discovered.

Ref: https://github.com/curl/curl/pull/13694#issuecomment-2120311894
Closes #13718
2024-05-28 08:12:00 +02:00
Viktor Szakats
1054c1cc20
cmake: always build unit tests with the testdeps target
Before this patch, the `testdeps` build target required `-DCURLDEBUG`
be set either via `ENABLE_DEBUG=ON` or `ENABLE_CURLDEBUG=ON` to build
the curl unit tests.

After fixing build issues in #13694, we can drop this requirement and
build unit tests unconditionally.

Depends-on: #13694
Depends-on: #13697 (fix unit test issue revealed by Old Linux CI job)
Follow-up to 39e7c22bb4 #11446
Closes #13698
2024-05-27 22:26:41 +02:00
Viktor Szakats
fc8e0dee30
build: untangle UNITTESTS and DEBUGBUILD macros
- fix `DEBUGBUILD` guards that should be `UNITTESTS`, in libcurl code
  used by unit tests.
- fix guards for libcurl functions used in unit tests only.
- sync `UNITTEST` attribute between declarations and definitions.
- drop `DEBUGBUILD` guard from test `unit2600`.
- fix guards for libcurl HSTS code used by both a unit test (`unit1660`)
  and `test0446`.
- update an existing AppVeyor CI job to test the issues fixed.

This fixes building tests with `CURLDEBUG` enabled but `DEBUGBUILD`
disabled. This can happen when building tests with CMake with
`ENABLE_DEBUG=ON` in Release config, or with `ENABLE_CURLDEBUG=ON`
and _without_ `ENABLE_DEBUG=ON`. Possibly also with autotools
when using `--enable-curldebug` without `--enable-debug`.

Test results:
- before:
  https://ci.appveyor.com/project/curlorg/curl/builds/49835609
  https://ci.appveyor.com/project/curlorg/curl/builds/49898529/job/k8qpbs8idby70smw
  https://github.com/curl/curl/actions/runs/9259078835/job/25470318167?pr=13798#step:13:821
- after: https://ci.appveyor.com/project/curlorg/curl/builds/49839255
  (the two failures are unrelated, subject to PR #13705)

Ref: #13592 (issue discovery)
Ref: #13689 (CI testing this PR with `DEBUGBUILD`/`CURLDEBUG` combinations)
Closes #13694
2024-05-27 21:15:50 +02:00
Viktor Szakats
0e176cabe4
cmake: whitespace, formatting/tidy-up in comments
Also correct casing in a few option descriptions.

Closes #13711
2024-05-27 18:07:10 +02:00
Viktor Szakats
a40204aff4
runtests: sort test IDs in summary lines
Changing this output:
```
TESTFAIL: These test cases failed: 2301 2303 2302 2307
```
Ref: https://github.com/curl/curl/actions/runs/9228638364/job/25393106631#step:6:21181

To:
```
TESTFAIL: These test cases failed: 2301 2302 2303 2307
```

Cherry-picked from #13766
Closes #13774
2024-05-26 22:43:05 +02:00
Stefan Eissing
e101a7a8b0
multi: add multi->proto_hash, a key-value store for protocol data
- add `Curl_hash_add2()` that passes a destructor function for
  the element added. Call element destructor instead of hash
  destructor if present.
- multi: add `proto_hash` for protocol related information,
  remove `struct multi_ssl_backend_data`.
- openssl: use multi->proto_hash to keep x509 shared store
- schannel: use multi->proto_hash to keep x509 shared store
- vtls: remove Curl_free_multi_ssl_backend_data() and its
  equivalents in the TLS backends

Closes #13345
2024-05-26 00:15:01 +02:00
Jan Venekamp
74e0bb1e7a
tests: add pytest for --ciphers and --tls13-ciphers options
Closes #13530
2024-05-25 23:57:26 +02:00
Orgad Shaneh
302bcd0838
tests: extend user/password parsing test1620
Closes #13756
2024-05-25 23:48:57 +02:00
Jan Venekamp
27ae00d230
tests: test_17_ssl_use.py clarify mbedtls TLSv1.3 support
Closes #13779
2024-05-25 23:43:50 +02:00
Stefan Eissing
17af2bca58
http: write last header line late
- HEADERFUNCTIONS might inspect response properties like
  CURLINFO_CONTENT_LENGTH_DOWNLOAD_T on seeing the last header line. If
  the line is being written before this is initialized, values are not
  available.

- write the last header line late when analyzing a HTTP response so that
  all information is available at the time of the writing.

- add test1485 to verify that CURLINFO_CONTENT_LENGTH_DOWNLOAD_T works
  on seeing the last header.

Fixes #13752
Reported-by: Harry Sintonen
Closes #13757
2024-05-25 23:42:53 +02:00
Dan Fandrich
548d169c9c tests: use exec when spawning nghttpx
This stops keeping perl and shell processes around that are no longer
needed, plus it eliminates an unneeded shell message when the server is
later terminated.

Closes #13772
2024-05-24 16:38:17 -07:00
Stefan Eissing
30de937bda
transfer: conn close on paused upload
- add 2 variations on test_07_42 which PAUSEs uploads
  and response connections terminating either right away
  or after the 100-continue response
- when detecting the connection being closed in transfer.c
  readwrite_data(), clear ALL send bits in data->req.keepon.
  It no longer makes send to wait for a KEEP_SEND_PAUSE or HOLD.
- in the protocol client writer add the check for incomplete
  response bodies. When an EOS is seen and the length is known,
  check that and fail if bytes are missing.

Reported-by: Sergey Bronnikov
Fixes #13740
Closes #13750
2024-05-23 23:55:09 +02:00
Stefan Eissing
5a913d8dc3
pytest: add ftp upload tests
- refs #13556
- allow anon uploads on vsftpd test server
- add test_30_05 for plain upload of 1k, 100k, 1m
- add test_31_05 for SSL upload of 1k, 100k, 1m
- verify file size and contents

Closes #13734
2024-05-22 09:12:43 +02:00
Stefan Eissing
f867942511
test: add test1546, chunked not last transfer encoding
with more than one transfer-encoding, 'chunked' must be the last added
to the writer stack (and therefore the first to decode). RFC 9112, ch.
6.1.

Closes #13736
2024-05-22 09:11:13 +02:00
Stefan Eissing
5e403dff06
test: add test1484, for HEAD with content
- test HEAD request with 'Transfer-Encoding:chunked' and
  non-encoded response content
- verifies #13725

Closes #13735
2024-05-22 09:10:00 +02:00
Viktor Szakats
dbd626ab82
tests: fix TFTP test 2305 on Windows
Ref: #13692
Closes #13724
2024-05-20 21:20:02 +02:00
Stefan Eissing
345557248e
pytest: fixes for recent python, add FTP tests
Fixes:
- in uds tests, abort also silently on os errors
- be conservative on the h3 goaway duration
- detect curl debug build and use in checks
- fix caddy version check for slight difference under linux
- set caddy default path fitting for linux
- fix deprecation warnings in valid time checks

FTP tests:
- add '--with-test-vsftpd=path' to configure
- use vsftpd default path suitable for linux
- add test_30 with plain FTP tests
- add test_31 with --ssl-reqd FTP tests
- add vsftpd to linux GHA for pytest workflows

Closes #13661
2024-05-17 16:53:17 +02:00
Stefan Eissing
dad8c1e305
pytest: add DELETE tests, check server version
- add tests for DELETE working
- check apache version in keepalive test
- fix some comments

Closes #13679
2024-05-16 22:51:25 +02:00
Daniel Stenberg
9b802e2d19
libtest: 2308 verifies CURLE_WRITE_ERROR after write callback error
Verifies that the issue in #13669 actually is fixed. This return code is
what the CURLOPT_WRITEFUNCTION manpage documents should be returned.

This code is mostly from the
Source-written-by: Trumeet on github
Closes #13671
2024-05-16 13:46:52 +02:00
Viktor Szakats
4eb4d660d6
tidy-up: whitespace [ci skip] 2024-05-14 16:49:47 +02:00
Stefan Eissing
22d8ce1970
http tests: in CI skip test_02_23* for quiche
For unknown reasons, these tests fail in CI often, but run fine locally.
Skip them in CI to avoid unrelated PRs to have failures.

Closes #13638
2024-05-14 16:19:42 +02:00
Viktor Szakats
17e51d2ec4
tests: fix test 1167 to skip digit-only symbols
This avoids mistaking symbols with their numeric value when using
certain C preprocessors which output these numeric values at the
beginning of the line as part of an expression.

Seen on OpenBSD 7.5 + clang.

Example `test1167.pl -v` output, before this patch:
```
Source: cpp /home/runner/work/curl/curl/tests/../include/curl/curl.h
Symbol: 20000
Line #3835:   20000 +  142,
[...]
Bad symbols in public header files:
   20000
   [...]
```
Ref: https://github.com/curl/curl/actions/runs/9069136530/job/24918015357#step:3:7513

Ref: #13583
Closes #13634
2024-05-14 09:58:34 +02:00
Viktor Szakats
0c49ea4ff2
tests: tidy up types in server code
Cherry-picked from #13489
Closes #13610
2024-05-13 11:16:26 +02:00
Viktor Szakats
25cbc2f79a
tests: make the unit test result type CURLcode
Before this patch, the result code was a mixture of `int` and
`CURLcode`.

Also adjust casts and fix a couple of minor issues found along the way.

Cherry-picked from #13489
Closes #13600
2024-05-12 18:53:07 +02:00
Daniel Stenberg
0e37b42dc9
url: make parse_login_details use memdup0
Also make the user and password arguments mandatory, since all code
paths in libcurl used them anyway.

Adapted unit test case 1620 to the new rules.

Closes #13584
2024-05-10 22:53:12 +02:00
Stefan Eissing
bc6e3e6049
h3/ngtcp2: improve error handling
- identify ngtcp2 and nghttp3 error codes that are fatal
- close quic connection on fatal errors
- refuse further filter operations once connection is closed
- confusion about the nghttp3 API. We should close the QUIC stream on
  cancel and not use the nghttp3 calls intended to be invoked when the
  QUIC stream was closed by the peer.

Closes #13562
2024-05-10 09:29:19 +02:00
Stefan Eissing
3ecba2decd
unit2604: use alloc instead of overlong string const
Closes #13563
2024-05-08 23:30:43 +02:00
MAntoniak
1ea7dce08d
mbedtls: support TLS 1.3
Closes #13539
2024-05-08 11:11:45 +02:00
Daniel Stenberg
cd3463d99e
curl_path: make Curl_get_pathname use dynbuf
... instead of malloc and memcpy

- unit test 2604 verifies Curl_get_pathname()

Closes #13550
2024-05-08 10:10:12 +02:00
Daniel Stenberg
c294f9cb56
lib: make protocol handlers store scheme name lowercase
- saves a lowercase operation when the "[scheme]_proxy" name is
  generated
- appears less "shouting"
- update test 970, 972, 1438 and 1536

Closes #13553
2024-05-08 09:39:30 +02:00
Daniel Gustafsson
46d7214ca9 tls: Remove EXAMPLEs from deprecated options
CURLOPT_EGDSOCKET and CURLOPT_RANDOM_FILE are both completely dead
so remove their example sections since the code there is useless.
There is still a way to inject a random file for OpenSSL older than
1.1.0 but it's not what the example showed (and it's not even done
with this option) so we refrain from documenting it here.

Closes: #13540
Reviewed-by: Daniel Stenberg <daniel@haxx.se>
2024-05-06 20:55:27 +02:00
Daniel Gustafsson
26dd9f0c8c tests: Only require EXAMPLE for non-deprecated options
Manpages which document deprecated CURLOPT_ or CURLINFO_ are not
required to have an EXAMPLE section since they might effectively
be dead no-ops which we don't want to trick users into believing
they can use by copying example code.

Closes: #13540
Reviewed-by: Daniel Stenberg <daniel@haxx.se>
2024-05-06 20:55:00 +02:00
Daniel Gustafsson
96852a130a tests: Mark tftpd timer function as noreturn
This avoids the below compiler warning:

tftpd.c:280:1: warning: function 'timer' could be declared with
    attribute 'noreturn' [-Wmissing-noreturn]

Closes: #13534
Reviewed-by: Daniel Stenberg <daniel@haxx.se>
2024-05-06 09:51:16 +02:00
Jan Venekamp
6e4b7abfd3
bearssl: use common code for cipher suite lookup
Take advantage of the Curl_cipher_suite_walk_str() and
Curl_cipher_suite_get_str() functions introduced in commit fba9afeb.

This also fixes CURLOPT_SSL_CIPHER_LIST not working at all for bearssl
due to commit ff74cef5.

Closes #13464
2024-05-02 13:17:02 +02:00
Stefan Eissing
b06619d0a3
tests: add SNI and peer name checks
- connect to DNS names with trailing dot
- connect to DNS names with double trailing dot
- rustls, always give `peer->hostname` and let it
  figure out SNI itself
- add SNI tests for ip address and localhost
- document in code and TODO that QUIC with ngtcp2+wolfssl
  does not do proper peer verification of the certificate
- mbedtls, skip tests with ip address verification as not
  supported by the library

Closes #13486
2024-04-29 08:20:35 +02:00
Stefan Eissing
c8e0cd1de8
http3: quiche+ngtcp2 improvements
- quiche: error transfers that try to receive on a closed
  or draining connection
- ngtcp2: use callback for extending max bidi streams. This
  allows more precise calculation of MAX_CONCURRENT as we
  only can start a new stream when the server acknowledges
  the close - not when we locally have closed it.
- remove a fprintf() from h2-download client to avoid excess
  log files on tests timing out.

Closes #13475
2024-04-26 14:03:05 +02:00
Stefan Eissing
fb22459dc1
vtls: TLS session storage overhaul
- add session with destructor callback
- remove vtls `session_free` method
- let `Curl_ssl_addsessionid()` take ownership
  of session object, freeing it also on failures
- change tls backend use
- test_17, add tests for SSL session resumption

Closes #13386
2024-04-26 13:58:36 +02:00
Stefan Eissing
2d2c27e5a3
multi: multi_wait improvements
- only call `multi_getsock()` once for all transfers
 - realloc pollset array on demand
 - fold repeated sockets

Closes #13150
2024-04-25 23:31:59 +02:00
Stefan Eissing
886899143f
content_encoding: ignore duplicate chunked encoding
- ignore duplicate "chunked" transfer-encodings from
  a server to accomodate for broken implementations
- add test1482 and test1483

Reported-by: Mel Zuser
Fixes #13451
Closes #13461
2024-04-25 17:50:16 +02:00
Jan Venekamp
fba9afebba
mbedTLS: implement CURLOPT_SSL_CIPHER_LIST option
Use a lookup list to set the cipher suites, allowing the
ciphers to be set by either openssl or IANA names.

To keep the binary size of the lookup list down we compress
each entry in the cipher list down to 2 + 6 bytes using the
C preprocessor.

Closes #13442
2024-04-24 14:30:37 +02:00
Dan Fandrich
7237ee2c3d tests: fix feature case in test1481
This test was being skipped everywhere because the feature never
matched.

Closes #13445
2024-04-23 08:46:17 -07:00
Gusted
00bef95946
tool_operate: don't truncate the etag save file by default
This fixes a regression of 75d79a4486. The
code in tool-operate truncated the etag save file, under the assumption
that the file would be written with a new etag value. However since
75d79a4486 that might not be the case
anymore and could result in the file being truncated when --etag-compare
and --etag-save was used and that the etag value matched with what the
server responded. Instead the truncation should not be done when a new
etag value should be written.

Test 3204 was added to verify that the file with the etag value doesn't
change the contents when used by --etag-compare and --etage-save and
that value matches with what the server returns on a non 2xx response.

Closes #13432
2024-04-23 11:33:20 +02:00
Abdullah Alyan
f8011ffa1e
tests: enable test 1117 for hyper
Closes #13436
2024-04-22 13:30:14 +02:00
Daniel Stenberg
fe17c162d0
urlapi: allow setting port number zero
Also set and check errno when strtoul() parsing numbers for better error
checking.

Updated test 1560

Closes #13427
2024-04-19 23:54:21 +02:00
Daniel Stenberg
d54b0adbad
test2406: verify -f with HTTP/2 2024-04-19 23:45:16 +02:00
Stefan Eissing
5c59f91427
http2 + ngtcp2: pass CURLcode errors from callbacks
- errors returned by Curl_xfer_write_resp() and the header variant are
  not errors in the protocol. The result needs to be returned on the
  next recv() from the protocol filter.

- make xfer write errors for response data cause the stream to be
  cancelled

- added pytest test_02_14 and test_02_15 to verify that also for
  parallel processing

Reported-by: Laramie Leavitt
Fixes #13411
Closes #13424
2024-04-19 23:45:16 +02:00