Commit Graph

30917 Commits

Author SHA1 Message Date
Wyatt O'Day
e92edfbef6
lib: add ability to disable auths individually
Both with configure and cmake

Closes #11490
2023-09-07 17:45:06 +02:00
Stefan Eissing
33dac9dfac
ngtcp2: fix handling of large requests
- requests >64K are send in parts to the filter
- fix parsing of the request to assemble it correctly
  from several sends
- open a QUIC stream only when the complete request has
  been collected

Closes #11815
2023-09-07 17:32:47 +02:00
Stefan Eissing
c849062677
openssl: when CURLOPT_SSL_CTX_FUNCTION is registered, init x509 store before
- we delay loading the x509 store to shorten the handshake time.
  However an application callback installed via CURLOPT_SSL_CTX_FUNCTION
  may need to have the store loaded and try to manipulate it.
- load the x509 store before invoking the app callback

Fixes #11800
Reported-by: guoxinvmware on github
Cloes #11805
2023-09-07 16:18:48 +02:00
Daniel Stenberg
25907fd5ba
krb5: fix "implicit conversion loses integer precision" warnings
conversions to/from enum and unsigned chars

Closes #11814
2023-09-07 16:17:13 +02:00
Stefan Eissing
3b30cc1a0d
pytest: improvements
- set CURL_CI for pytest runs in CI environments
- exclude timing sensitive tests from CI runs
- for failed results, list only the log and stat of
  the failed transfer

- fix type in http.c comment

Closes #11812
2023-09-07 10:30:14 +02:00
Stefan Eissing
108e51835e
CI: move on to ngtcp2 v0.19.1
Closes #11809
2023-09-06 23:11:30 +02:00
Dan Fandrich
2683de3078 CI: run Circle macOS builds on x86 for now
The ARM machines aren't ready for us and requesting them now causes
warnings e-mails to be sent to some PR pushers.

Ref: #11771
2023-09-05 22:53:40 -07:00
Viktor Szakats
c63a4b6544
http3: adjust cast for ngtcp2 v0.19.0
ngtcp2 v0.19.0 made size of `ecn` member of `ngtcp2_pkt_info`
an `uint8_t` (was: `uint32_t`). Adjust our local cast accordingly.

Fixes:
```
./curl/lib/vquic/curl_ngtcp2.c:1912:12: warning: implicit conversion loses integer precision: 'uint32_t' (aka 'unsigned int') to 'uint8_t' (aka 'unsigned char') [-Wimplicit-int-conversion]
  pi.ecn = (uint32_t)ecn;
         ~ ^~~~~~~~~~~~~
```

Also bump ngtcp2, nghttp3 and nghttp2 to their latest versions in our
docs and CI.

Ref: 80447281bb
Ref: https://github.com/ngtcp2/ngtcp2/pull/877
Closes #11798
2023-09-05 15:25:37 +00:00
Stefan Eissing
2485547da0
http: fix sending of large requests
- refs #11342 where errors with git https interactions
  were observed
- problem was caused by 1st sends of size larger than 64KB
  which resulted in later retries of 64KB only
- limit sending of 1st block to 64KB
- adjust h2/h3 filters to cope with parsing the HTTP/1.1
  formatted request in chunks

- introducing Curl_nwrite() as companion to Curl_write()
  for the many cases where the sockindex is already known

Fixes #11342 (again)
Closes #11803
2023-09-05 16:36:03 +02:00
Stefan Eissing
a8a82140aa
pytest: fix check for slow_network skips to only apply when intended
Closes #11801
2023-09-05 14:10:01 +02:00
Daniel Stenberg
dd5999b686
curl_url_get/set.3: add missing semicolon in SYNOPSIS 2023-09-05 11:30:53 +02:00
Daniel Stenberg
a8f21bb40b
CURLOPT_URL.3: explain curl_url_set() uses the same parser 2023-09-05 11:16:39 +02:00
Daniel Stenberg
cb1197cbfb
CURLOPT_URL.3: add two URL API calls in the see-also section 2023-09-05 10:34:42 +02:00
Dan Fandrich
22b1f7d032 CI: add a 32-bit i686 Linux build
This is done by cross-compiling under regular x86_64 Linux.  Since the
kernel offers backwards compatibility, the binaries can be tested as
normal.

Closes #11799
2023-09-04 17:11:44 -07:00
Dan Fandrich
876278935f tests: fix a type warning on 32-bit x86 2023-09-04 17:10:07 -07:00
Viktor Szakats
205d686ce1
tests: delete stray .orig file
Follow-up to 331b89a319
Closes #11797
2023-09-04 22:20:16 +00:00
Daniel Stenberg
78e133e942
RELEASE-NOTES: synced 2023-09-04 22:29:18 +02:00
Viktor Szakats
3a6287d726
lib: silence compiler warning in inet_ntop6
```
./curl/lib/inet_ntop.c:121:21: warning: possible misuse of comma operator here [-Wcomma]
        cur.base = i, cur.len = 1;
                    ^
./curl/lib/inet_ntop.c:121:9: note: cast expression to void to silence warning
        cur.base = i, cur.len = 1;
        ^~~~~~~~~~~~
        (void)(     )
```

Closes #11790
2023-09-04 18:56:49 +00:00
Daniel Stenberg
95a865b462
transfer: also stop the sending on closed connection
Previously this cleared the receiving bit only but in some cases it is
also still sending (like a request-body) when disconnected and neither
direction can continue then.

Fixes #11769
Reported-by: Oleg Jukovec
Closes #11795
2023-09-04 19:51:59 +02:00
John Bampton
356567c649
docs: change sub-domain to subdomain
https://en.wikipedia.org/wiki/Subdomain

Closes #11793
2023-09-04 19:51:27 +02:00
Stefan Eissing
519b1cf97b
multi: more efficient pollfd count for poll
- do not use separate pollfds for sockets that have POLLIN+POLLOUT

Closes #11792
2023-09-04 19:50:22 +02:00
Stefan Eissing
331b89a319
http2: polish things around POST
- added test cases for various code paths
- fixed handling of blocked write when stream had
  been closed inbetween attempts
- re-enabled DEBUGASSERT on send with smaller data size

- in debug builds, environment variables can be set to simulate a slow
  network when sending data. cf-socket.c and vquic.c support
  * CURL_DBG_SOCK_WBLOCK: percentage of send() calls that should be
    answered with a EAGAIN. TCP/UNIX sockets.
    This is chosen randomly.
  * CURL_DBG_SOCK_WPARTIAL: percentage of data that shall be written
    to the network. TCP/UNIX sockets.
    Example: 80 means a send with 1000 bytes would only send 800
    This is applied to every send.
  * CURL_DBG_QUIC_WBLOCK: percentage of send() calls that should be
    answered with EAGAIN. QUIC only.
    This is chosen randomly.

Closes #11756
2023-09-04 19:48:49 +02:00
Daniel Stenberg
c9260cf9fe
docs: add curl_global_trace to some SEE ALSO sections
Closes #11791
2023-09-04 09:54:10 +02:00
Daniel Stenberg
8c040ca3d3
os400: fix checksrc nits
Closes #11789
2023-09-03 19:12:28 +02:00
Nicholas Nethercote
50aa325742
hyper: remove hyptransfer->endtask
`Curl_hyper_stream` needs to distinguish between two kinds of
`HYPER_TASK_EMPTY` tasks: (a) the `foreach` tasks it creates itself, and
(b) background tasks that hyper produces. It does this by recording the
address of any `foreach` task in `hyptransfer->endtask` before pushing
it into the executor, and then comparing that against the address of
tasks later polled out of the executor.

This works right now, but there is no guarantee from hyper that the
addresses are stable. `hyper_executor_push` says "The executor takes
ownership of the task, which should not be accessed again unless
returned back to the user with `hyper_executor_poll`". That wording is a
bit ambiguous but with my Rust programmer's hat on I read it as meaning
the task returned with `hyper_executor_poll` may be conceptually the
same as a task that was pushed, but that there are no other guarantees
and comparing addresses is a bad idea.

This commit instead uses `hyper_task_set_userdata` to mark the `foreach`
task with a `USERDATA_RESP_BODY` value which can then be checked for,
removing the need for `hyptransfer->endtask`. This makes the code look
more like that hyper C API examples, which use userdata for every task
and never look at task addresses.

Closes #11779
2023-09-03 18:42:37 +02:00
Dave Cottlehuber
a86fcb284f
ws: fix spelling mistakes in examples and tests
Closes #11784
2023-09-03 18:01:58 +02:00
Daniel Stenberg
dffd996e3b
tool_filetime: make -z work with file dates before 1970
Fixes #11785
Reported-by: Harry Sintonen
Closes #11786
2023-09-03 17:45:19 +02:00
Dan Fandrich
5949369c9f build: fix portability of mancheck and checksrc targets
At least FreeBSD preserves cwd across makefile lines, so rules
consisting of more than one "cd X; do_something" must be explicitly run
in a subshell to avoid this. This problem caused the Cirrus FreeBSD
build to fail when parallel make jobs were enabled.
2023-09-01 15:08:39 -07:00
Dan Fandrich
78e0938fcc CI: adjust labeler match patterns for new & obsolete files 2023-09-01 15:08:39 -07:00
Dan Fandrich
a20fbb0348 configure: trust pkg-config when it's used for zlib
The library flags retrieved from pkg-config were later thrown out and
harded-coded, which negates the whole reason to use pkg-config.
Also, previously, the assumption was made that --libs-only-l and
--libs-only-L are the full decomposition of --libs, which is untrue and
would not allow linking against a static zlib. The new approach is
better in that it uses --libs, although only if --libs-only-l returns
nothing.

Bug: https://curl.se/mail/lib-2023-08/0081.html
Reported-by: Randall
Closes #11778
2023-09-01 15:08:39 -07:00
Stefan Eissing
280f90061a
CI/ngtcp2: clear wolfssl for when cache is ignored
Closes #11783
2023-09-01 23:48:47 +02:00
Daniel Stenberg
cb5e18100a
RELEASE-NOTES: synced 2023-09-01 14:59:50 +02:00
Nicholas Nethercote
73f4ef584f
hyper: fix a progress upload counter bug
`Curl_pgrsSetUploadCounter` should be a passed a total count, not an
increment.

This changes the failing diff for test 579 with hyper from this:
```
 Progress callback called with UL 0 out of 0[LF]
-Progress callback called with UL 8 out of 0[LF]
-Progress callback called with UL 16 out of 0[LF]
-Progress callback called with UL 26 out of 0[LF]
-Progress callback called with UL 61 out of 0[LF]
-Progress callback called with UL 66 out of 0[LF]
+Progress callback called with UL 29 out of 0[LF]
```
to this:
```
 Progress callback called with UL 0 out of 0[LF]
-Progress callback called with UL 8 out of 0[LF]
-Progress callback called with UL 16 out of 0[LF]
-Progress callback called with UL 26 out of 0[LF]
-Progress callback called with UL 61 out of 0[LF]
-Progress callback called with UL 66 out of 0[LF]
+Progress callback called with UL 40 out of 0[LF]
```
Presumably a step in the right direction.

Closes #11780
2023-09-01 14:44:12 +02:00
Daniel Stenberg
b8dabfb1e1
awssiv4: avoid freeing the date pointer on error
Since it was not allocated, don't free it even if it was wrong syntax

Bug: https://bugs.chromium.org/p/oss-fuzz/issues/detail?id=61908

Follow-up to b137634ba3

Closes #11782
2023-09-01 13:36:10 +02:00
Stefan Eissing
289cd4073c
CI: ngtcp2-linux: use separate caches for tls libraries
allow ever changing master for wolfssl

Closes #11766
2023-09-01 13:35:00 +02:00
Stefan Eissing
1c9781475c replace master as wolfssl-version with recent commit 2023-09-01 11:43:55 +02:00
Stefan Eissing
40f3ee0db3 wolfssl, use master again in CI
- with the shared session update fix landed in master, it
  is time to use that in our CI again
2023-09-01 11:43:55 +02:00
Nicholas Nethercote
49664d6afc tests: fix formatting errors in FILEFORMAT.md.
Without the surrounding backticks, these tags get swallowed when the
markdown is rendered.

Closes #11777
2023-08-31 16:50:44 -07:00
Viktor Szakats
26c7feb8b9
cmake: add support for CURL_DEFAULT_SSL_BACKEND
Allow overriding the default TLS backend via a CMake setting.

E.g.:
`cmake [...] -DCURL_DEFAULT_SSL_BACKEND=mbedtls`

Accepted values: bearssl, gnutls, mbedtls, openssl, rustls,
schannel, secure-transport, wolfssl

The passed string is baked into the curl/libcurl binaries.
The value is case-insensitive.

We added a similar option to autotools in 2017 via
c7170e20d0.

TODO: Convert to lowercase to improve reproducibility.

Closes #11774
2023-08-31 23:04:05 +00:00
Viktor Szakats
4727d32e2e
sectransp: fix compiler warnings
https://github.com/curl/curl-for-win/actions/runs/6037489221/job/16381860220#step:3:11046
```
/Users/runner/work/curl-for-win/curl-for-win/curl/lib/vtls/sectransp.c:2435:14: warning: unused variable 'success' [-Wunused-variable]
    OSStatus success;
             ^
/Users/runner/work/curl-for-win/curl-for-win/curl/lib/vtls/sectransp.c:3300:44: warning: unused parameter 'sha256len' [-Wunused-parameter]
                                    size_t sha256len)
                                           ^
```

Closes #11773
2023-08-31 23:02:43 +00:00
Viktor Szakats
ce3dce9015
tidy-up: mostly whitespace nits
- delete completed TODO from `./CMakeLists.txt`.
- convert a C++ comment to C89 in `./CMake/CurlTests.c`.
- delete duplicate EOLs from EOF.
- add missing EOL at EOF.
- delete whitespace at EOL (except from expected test results).
- convert tabs to spaces.
- convert CRLF EOLs to LF in GHA yaml.
- text casing fixes in `./CMakeLists.txt`.
- fix a codespell typo in `packages/OS400/initscript.sh`.

Closes #11772
2023-08-31 23:02:10 +00:00
Dan Fandrich
63f23fafc9 CI: remove Windows builds from Cirrus, without replacement
If we don't do this, all coverage on Cirrus will cease in a few days. By
removing the Windows builds, the FreeBSD one should still continue
as before. The Windows builds will need be moved to another service to
maintain test coverage.

Closes #11771
2023-08-31 15:18:28 -07:00
Dan Fandrich
3262ce6f28 CI: switch macOS ARM build from Cirrus to Circle CI
Cirrus is drastically reducing their free tier on Sept. 1, so they will
no longer perform all these builds for us. All but one build has been
moved, with the LibreSSL one being dropped because of linking problems
on Circle.

One important note about this change is that Circle CI is currently
directing all these builds to x86_64 hardware, despite them requesting
ARM. This is because ARM nodes are scheduled to be available on the
free tier only in December. This reduces our architectural diversity
until then but it should automatically come back once those machines are
enabled.
2023-08-31 15:18:28 -07:00
Dan Fandrich
a7f4e90ce7 CI: use the right variable for BSD make
BSD uses MAKEFLAGS instead of MAKE_FLAGS so it wasn't doing parallel
builds before.
2023-08-31 15:18:21 -07:00
Dan Fandrich
756de05d89 CI: drop the FreeBSD 12.X build
Cirrus' new free tier won't let us have many builds, so drop the
nonessential ones. The FreeBSD 13.X build will still give us the most
relevant FreeBSD coverage.
2023-08-30 23:50:18 -07:00
Dan Fandrich
cc746076e0 CI: move the Alpine build from Cirrus to GHA
Cirrus is reducing their free tier to next to nothing, so we must move
builds elsewhere.
2023-08-30 23:50:16 -07:00
Stefan Eissing
3f283e90c3 test_07_upload.py: fix test_07_34 curl args
- Pass correct filename to --data-binary.

Prior to this change --data-binary was passed an incorrect filename due
to a missing separator in the arguments list. Since aacbeae7 curl will
error on incorrect filenames for POST.

Fixes https://github.com/curl/curl/issues/11761
Closes https://github.com/curl/curl/pull/11763
2023-08-30 11:32:15 -04:00
Nicholas Nethercote
9254c7b344
tests: document which tests fail due to hyper's lack of trailer support.
Closes #11762
2023-08-30 11:41:04 +02:00
Nicholas Nethercote
e5e370965f
docs: removing "pausing transfers" from HYPER.md.
It's a reference to #8600, which was fixed by #9070.

Closes #11764
2023-08-30 11:40:22 +02:00
Patrick Monnerat
fe58e2945b
os400: handle CURL_TEMP_PRINTF() while building bind source
Closes #11547
2023-08-30 11:39:01 +02:00