Commit Graph

32191 Commits

Author SHA1 Message Date
Daniel Stenberg
16bad89239
RELEASE-NOTES: synced 2024-04-12 13:25:44 +02:00
Stefan Eissing
8482ce53dd
CURLINFO_REQUEST_SIZE: fixed, add tests for transfer infos reported
- tests for 'size_request' and other stats reported, for
  presence and consistency

Reported-by: Jonatan Vela
Fixes #13269
Closes #13275
2024-04-12 13:20:09 +02:00
Viktor Szakats
0a4419ae4c
dist: add files missing from release tarball
Closes #13346
2024-04-11 15:52:08 +00:00
Viktor Szakats
bba4c313bc
ci: parallelize more, tidy up cmake commands (distcheck, macos)
Also enable `-DCURL_WERROR=ON` in the Linux cmake build test.

Closes #13343
2024-04-11 15:52:07 +00:00
Toon Claes
0f7be5a519
docs: add CURLOPT_NOPROGRESS to CURLOPT_XFERINFOFUNCTION example
It's important to set `CURLOPT_NOPROGRESS` to `0` if you want your
transfer callback function, set by `CURLOPT_XFERINFOFUNCTION`, getting
called. To emphasize this to the users, add this to the code example.

Closes #13348
2024-04-11 15:59:53 +02:00
RainRat
1087937992
misc: fix typos
Closes #13344
2024-04-11 15:44:22 +02:00
Colin Leroy-Mira
bfe54b0e88
file: add support for getting basic directory listings
Not supported on Windows (yet)

Closes #13137
2024-04-11 12:37:12 +02:00
Viktor Szakats
e14daeb8a4
ci: add curl-for-win builds: Linux MUSL, macOS, Windows
Linux MUSL (llvm/clang), macOS Apple clang, Windows (llvm/clang).

Configured with HTTP/2 and HTTP/3 and other dependencies (the default
curl-for-win) for a comprehensive build test.

```
curl 8.8.0-DEV (x86_64-unknown-linux-musl) libcurl/8.8.0-DEV LibreSSL/3.9.1 zlib/1.3.1 brotli/1.1.0 zstd/1.5.6 libpsl/0.21.5 libssh2/1.11.0 nghttp2/1.61.0 ngtcp2/1.4.0 nghttp3/1.2.0
Protocols: dict file ftp ftps gopher gophers http https imap imaps ipfs ipns mqtt pop3 pop3s rtsp scp sftp smb smbs smtp smtps telnet tftp ws wss
Features: alt-svc AsynchDNS brotli HSTS HTTP2 HTTP3 HTTPS-proxy IPv6 Largefile libz NTLM PSL SSL threadsafe UnixSockets zstd

curl 8.8.0-DEV (x86_64-apple-darwin) libcurl/8.8.0-DEV LibreSSL/3.9.1 zlib/1.3.1 brotli/1.1.0 zstd/1.5.6 libpsl/0.21.5 libssh2/1.11.0 nghttp2/1.61.0 ngtcp2/1.4.0 nghttp3/1.2.0
Protocols: dict file ftp ftps gopher gophers http https imap imaps ipfs ipns ldap ldaps mqtt pop3 pop3s rtsp scp sftp smb smbs smtp smtps telnet tftp ws wss
Features: alt-svc AsynchDNS brotli HSTS HTTP2 HTTP3 HTTPS-proxy IPv6 Largefile libz NTLM PSL SSL threadsafe UnixSockets zstd

curl 8.8.0-DEV (x86_64-w64-mingw32) libcurl/8.8.0-DEV LibreSSL/3.9.1 zlib/1.3.1 brotli/1.1.0 zstd/1.5.6 WinIDN libpsl/0.21.5 libssh2/1.11.0 nghttp2/1.61.0 ngtcp2/1.4.0 nghttp3/1.2.0
Protocols: dict file ftp ftps gopher gophers http https imap imaps ipfs ipns ldap ldaps mqtt pop3 pop3s rtsp scp sftp smb smbs smtp smtps telnet tftp ws wss
Features: alt-svc AsynchDNS brotli HSTS HTTP2 HTTP3 HTTPS-proxy IDN IPv6 Kerberos Largefile libz NTLM PSL SPNEGO SSL SSPI threadsafe UnixSockets zstd
```

Limited to x64, because for build testing the additional CPUs don't add
much value compared to the extra build time. They can be enabled easily
if deemed useful.

To the extent of curl-for-win configuration options, it's trivial to add
further build combinations.

Closes #13335
2024-04-11 07:48:52 +00:00
Viktor Szakats
8a622baf9e
OS400: fix shellcheck warnings in scripts
- use `$()` instead of backticks, and re-arrange double-quotes inside.
- add missing `|| exit 1` to `cd` calls. (could be dropped by using `set -eu`.)
- add `-n` to a few `if`s.
- shorten redirections by using `{} >` (as shellcheck recommended).
- silence warnings where variables were detected as unused (SC2034).
- a couple misc updates to silence warnings.
- switch to bash shebang for `-ot` feature.
- split two lines to unbreak syntax highlighting in my editor. (`$(expr \`, `$(dirname \`)

Also enable CI checks for OS/400 shell scripts.

Ref: #13307
Closes #13309
2024-04-11 07:48:51 +00:00
Stefan Eissing
8dd81bd5db
lib: add Curl_xfer_write_resp_hd
Add method in protocol handlers to allow writing of a single,
0-terminated header line. Avoids parsing and copying these lines.

Closes #13165
2024-04-11 09:29:21 +02:00
Stefan Eissing
c296abd42d
llist: add Curl_llist_append()
- use for better readability in all places where the "insert_next"
  actually performs an append to the list
- add some tests in unit1300

Closes #13336
2024-04-11 09:00:51 +02:00
Stefan Eissing
8cee4c9238
gnutls: lazy init the trust settings
- delay loading of trust anchors and CRLs after the ClientHello
  has been sent off
- add tracing to IO operations
- on IO errors, return the CURLcode of the underlying filter

Closes #13339
2024-04-11 08:59:25 +02:00
Marcel Raad
61e6db87ac
http_negotiate: fix CURL_DISABLE_PROXY build
`proxyuserpwd` was removed from `dynamically_allocated_data` in commit
f46385d36d.

Closes https://github.com/curl/curl/pull/13334
2024-04-10 14:56:05 +02:00
Viktor Szakats
4f15443d2c
quic: fixup duplicate static function name (for cmake unity)
Visible in daily curl-for-win builds:
https://github.com/curl/curl-for-win/actions/runs/8621925870

```
lib/vquic/curl_ngtcp2.c:1916:12: error: redefinition of 'ossl_new_session_cb'
static int ossl_new_session_cb(SSL *ssl, SSL_SESSION *ssl_sessionid)
           ^
lib/vtls/openssl.c:2978:12: note: previous definition is here
static int ossl_new_session_cb(SSL *ssl, SSL_SESSION *ssl_sessionid)
           ^
```
https://github.com/curl/curl-for-win/actions/runs/8621925870/job/23631885439#step:3:6965

Follow-up to 3210101088 #13172
Closes #13332
2024-04-10 10:24:21 +00:00
Viktor Szakats
b445818586
appveyor: make VS2010 job build-only, enable Schannel, fix compiler warnings
Tests were consistently flaky for a while.

Also fix compiler warnings in `CertOpenStore()` calls for old MSVC compilers:
```
C:/projects/curl/lib/vtls/schannel.c(688):
  warning C4306: 'type cast' : conversion from 'int' to 'LPCSTR' of greater size
C:/projects/curl/lib/vtls/schannel_verify.c(642):
  warning C4306: 'type cast' : conversion from 'int' to 'LPCSTR' of greater size
```
Ref: https://ci.appveyor.com/project/curlorg/curl/builds/49580310/job/ywu2y44kymgc0nif#L106

Closes #13330
2024-04-10 07:32:38 +00:00
Daniel Stenberg
6389ba87b8
projects: drop MSVC project files for recent versions
We encourage users to generate visual studio project files using CMake.

We keep project files in git for ancient visual studio versions that
cmake cannot generate files for, but we no longer ship the project files
in the tarballs.

appveyor: switch VisualStudioSolution job to VC12 (Visual Studio 2013)

Co-Authored-by: Viktor Szakats
Co-Authored-by: Jay Satiro

Closes #13311
2024-04-10 07:55:24 +02:00
Viktor Szakats
3d569aaaf8
cmake: use namespaced custom target names
Rename custom target to namespaced (unique) names to avoid colliding
with 3rd-party projects (e.g. libzip) built together with curl.

Reported-by: hammlee96 on github
Fixes #13324
Closes #13326
2024-04-09 21:20:54 +00:00
Viktor Szakats
feb1a3527e
appveyor: re-enable OpenSSL 3, bump to 3.2.1
Ref: b62454a875 #13266
Closes #13329
2024-04-09 21:16:53 +00:00
Stefan Eissing
a6ef4056e8
CI: upgrade openssl version to 3.3.0 for openssl-quic
Closes #13328
2024-04-09 17:41:46 +02:00
Daniel Stenberg
6416192a8c
RELEASE-NOTES: synced
Bump to 8.8.0-DEV
2024-04-09 17:18:28 +02:00
Daniel Stenberg
187b23b0a4
curl_multi_waitfds.md: add protocol mention
Follow-up to 02beac6bb6
2024-04-09 16:59:28 +02:00
Dmitry Karpov
02beac6bb6
lib: add curl_multi_waitfds
New function call, similar to curl_multi_fdset()

Closes #13135
2024-04-09 16:53:40 +02:00
Viktor Szakats
e469ac4494
dist: verify tarball reproducibility in CI
Closes #13327
2024-04-09 14:31:23 +00:00
Stefan Eissing
28c5ddf13a
tests: stabilitze test_02_23*
- h2-download now always opens the output file on first write callback
  invocation, if it will pause the transfer or not.
- Checks on output files then does not depend on the amount of data curl
  has collected for the first write.

Closes #13323
2024-04-09 13:27:21 +02:00
Stefan Eissing
e7de80e8ce
tls: fix compile issues on old-linux CI
Follow-up to 3210101088
Closes #13325
2024-04-09 13:26:19 +02:00
Viktor Szakats
b6a4f9aa1e
dist: add reproducible dir entries to tarballs
In the initial implementation of reproducible tarballs, they were
missing directory entries, while .zip archives had them. It meant
that on extracting the tarball, on-disk directory entries got the
current timestamp.

This patch fixes this by including directory entries in the tarball,
with reproducible timestamps. It also moves sorting inside tar,
to ensure reproducible directory entry timestamps on extract
(without the need of `--delay-directory-restore` option, when
extracting with GNU tar. BSD tar got that right by default.)

GNU tar 1.28 (2014-07-28) introduced `--sort=`.

Ref: https://github.com/curl/curl/pull/13299#discussion_r1555957350
Follow-up to 860cd5fc2d #13299
Closes #13322
2024-04-09 08:37:57 +00:00
Stefan Eissing
3210101088
tls: use shared init code for TCP+QUIC
Closes #13172
2024-04-09 09:08:05 +02:00
Daniel Stenberg
49573bc187
.mailmap: update Gisle's preferred email 2024-04-09 08:50:07 +02:00
Jan Macku
6b24f099fc
doc: pytest --repeat -> --count
Pytest doesn't have a `--repeat` option, but it does have a `--count`
option.

```
--count=COUNT         Number of times to repeat each test
```

Closes #13218
2024-04-09 08:44:09 +02:00
Daniel Stenberg
b4e8183f52
src/Makefile.am: access curl.txt using a relative path, not abs
... to make it work when mounted using different mount points. Like when
generated/used inside and outside of a docker image.

Closes #13320
2024-04-09 08:10:20 +02:00
Daniel Stenberg
9eafc11552
build: remove MacOSX-Framework script
I don't think this is much used these days.

Also remove the libcurl.plist file used (only) by this script

Closes #13313
2024-04-08 22:46:51 +02:00
Daniel Stenberg
fd6c16c345
release-tools.sh: store the timestamp and release tag too
When maketgz invokes this script to generate the docs/RELEASE-TOOLS.md
file that gets bundled in the release, it now also passes on the exact
timestamp and version number so that those details also get mentioned in
the document. They will help users reproduce an identical tarball.

Closes #13319
2024-04-08 22:43:48 +02:00
Viktor Szakats
41e07d836f
GHA: disable permissions where missing
Reviewed-by: Daniel Stenberg
Closes #13306
2024-04-08 14:27:09 +00:00
Stefan Eissing
7f26fd17dd
CI: update component versions
- ngtcp2: v1.4.0
- nghttp3: v1.2.0
- nghttp2: v1.61.0
- mod_h2: v2.0.27

Closes #13316
2024-04-08 16:19:23 +02:00
Jérôme Leclercq
6b86471fe1
CMake: check fseeko after detecting HAVE_FILE_OFFSET_BITS
Closes #13264
2024-04-08 15:51:35 +02:00
Stefan Eissing
1302aa6b50
http2: emit RST when client write fails
- When the writing of response data fails, reset the stream
  and do not return a callback error to nghttp2. That would
  be a fatal error for the connection and harm other requests.
- add test cases for various abort scenarios

Reported-by: Konstantin Kuzov
Fixes #13292
Closes #13298
2024-04-08 15:49:52 +02:00
Kailun Qin
b679efc0bb
mbedtls: call mbedtls_ssl_setup() after RNG callback is set
Since mbedTLS v3.6.0, the RNG check added in ssl_conf_check() will fail
if no RNG is provided when calling mbedtls_ssl_setup().

Therefore, mbedtls_ssl_conf_rng() needs to be called before the SSL
context is passed to mbedtls_ssl_setup().

Ref: b422cab052

Signed-off-by: Kailun Qin <kailun.qin@intel.com>
Closes #13314
2024-04-08 14:27:12 +02:00
Daniel Stenberg
50def7c881
NTLM_WB: drop support
The feature has not worked for months and has been marked as DEPRECATED
for six+ months.

Closes #13249
2024-04-08 13:58:58 +02:00
Daniel Stenberg
9e848439d8
curl_trc: fix build error when lacking verbose messages
Follow-up from 0b28ece657
Closes #13312
2024-04-08 13:57:14 +02:00
Viktor Szakats
5adbf72b79
contrithanks: honor CURLWWW variable
Reviewed-by: Daniel Stenberg
Closes #13315
2024-04-08 11:56:41 +00:00
Viktor Szakats
fa69b41c77
GHA: add shellcheck job and fix warnings, shell tidy-ups
Reviewed-by: Daniel Stenberg
Closes #13307
2024-04-08 09:37:24 +00:00
Viktor Szakats
2c4f836f70
dist: do not require Perl in maketgz
Perl remains required for the tarball build process.

Follow-up to 860cd5fc2d #13299

Reviewed-by: Daniel Stenberg
Closes #13310
2024-04-08 09:28:32 +00:00
Daniel Stenberg
20434a6aeb
RELEASE-NOTES: synced 2024-04-08 10:33:36 +02:00
Daniel Stenberg
bcc2e90e45
docs/cmdline-opts: invoke managen using a relative path
... no need to use an absolute path, that makes the build unncessarily
fail if invoked using a different mount point. managen now takes options
to find the input files.

Update test1478 to provide the dir arguments to managen

Closes #13281
2024-04-08 10:27:27 +02:00
Daniel Stenberg
a3b084b913
GHA: add valgrind to a wolfSSL build
Closes #13274
2024-04-08 09:39:16 +02:00
Viktor Szakats
860cd5fc2d
dist: set -eu, fix shellcheck, make reproducible and smaller tarballs
- set bash `-eu` and fix fallouts.
- fix shellcheck warnings.
- set and use `SOURCE_DATE_EPOCH` for reproducibility.
  Authored-by: Daniel J. H.
  Ref: #13280
- set `TZ=UTC` and `LC_ALL=C` for reproducibility.
- make file timestamps in tarball/zip reproducible.
- make directory timestamps in zip reproducible.
- make timestamps of tarballs/zip reproducible.
- make file order in tarball/zip reproducible.
- omit extra file metadata from zip for reproducibility.
- use maximum zip compression.
- use POSIX `ustar` tarball format to avoid supply chain vulnerability:
  https://seclists.org/oss-sec/2021/q4/0
- make uid/gid in tarball reproducible.
- omit owner user/group names from tarball for reproducibility and privacy.
- omit current timestamp from .gz header for reproducibility.
- display SHA-256 hashes of produced tarballs/zip.
- fix whitespace.

`.tar.gz` also became smaller in the process: 4,462,311 -> 4,148,249 bytes (8.7.1)

Requires GNU tar, GNU date, `sha256sum`.

Reviewed-by: Daniel Stenberg
Ref: #13250
Closes #13299
2024-04-07 22:28:42 +00:00
Gisle Vanem
a42de088a2 tests/http: fix compiler warning
- Init result code variable to fix clang warning that it may be used
  uninitialized.

Fixes https://github.com/curl/curl/issues/13301
Closes https://github.com/curl/curl/pull/13304
2024-04-07 14:21:31 -04:00
Stefan Eissing
9287563e86 vquic: use new curl_int64_t type
- add curl_int64_t signed 64-bit type for lib use

- define CURL_PRId64, CURL_PRIu64 format ids

- use curl_int64_t in vquic

curl_int64_t signed complements the existing curl_uint64_t unsigned.

Note that `curl_int64_t` and `int64_t` are assignable from each other
but not identical. Some platforms with 64 long type defint int64_t as
"long long" (staring at macOS) which messes up things like pointers and
format identifiers.

Closes https://github.com/curl/curl/pull/13293
2024-04-06 19:08:12 -04:00
Jay Satiro
bf567dd9f1 lib: use multi instead of multi_easy for the active multi
- Use data->multi and not data->multi_easy to refer to the active multi.

The easy handle's active multi is always data->multi.

This is a follow up to 757dfdf which changed curl so that an easy handle
used with the easy interface and then multi interface cannot have two
different multi handles associated with it at the same time
(data->multi_easy from the easy interface and data->multi from the multi
interface).

Closes https://github.com/curl/curl/pull/12665
2024-04-05 16:47:36 -04:00
Viktor Szakats
dc178faee9
tidy-up: whitespace [ci skip] 2024-04-05 15:50:07 +00:00