Commit Graph

29496 Commits

Author SHA1 Message Date
Max Dymond
a4fd257c1b ci: Reuse fuzzing snippet from curl-fuzzer project 2022-12-02 22:22:06 +00:00
Diogo Teles Sant'Anna
a2f5a4ca6f
GHA: clarify workflows permissions, set least possible privilege
Set top-level permissions to None on all workflows, setting per-job
permissions. This avoids that new jobs inherit unwanted permissions.

Discussion: https://curl.se/mail/lib-2022-11/0028.html

Signed-off-by: Diogo Teles Sant'Anna <diogoteles@google.com>

Closes #9928
2022-12-02 23:13:54 +01:00
Viktor Szakats
8fc2423338
Makefile.mk: address minor issues
- Fix `NROFF` auto-detection with certain shell/make-build combinations:

  When a non-MSYS2 GNU Make runs inside an MSYS2 shell, Make executes
  the detection command as-is via `CreateProcess()`. It fails because
  `command` is an `sh` built-in. Ensure to explicitly invoke the shell.

- Initialize user-customizable variables:

  Silences a list of warnings when running GNU Make with the option
  `--warn-undefined-variables`. Another benefit is that it's now easy
  to look up all user-customizable `Makefile.mk` variables by grepping
  for ` ?=` in the curl source tree.

  Suggested-by: Gisle Vanem
  Ref: https://github.com/curl/curl/pull/9764#issuecomment-1330674433

- Fix `MKDIR` invocation:

  Avoid a warning and potential issue in envs without forward-slash
  support.

Closes #10000
2022-12-02 15:20:41 +00:00
Rob de Wit
73c4f9696a
curl_get_line: allow last line without newline char
improve backwards compatibility

Test 3200 verifies

Closes #9973
2022-12-02 11:28:43 +01:00
Daniel Stenberg
73d6f41489
cookie: open cookie jar as a binary file
On Windows there is a difference and for text files, ^Z means end of
file which is not desirable.

Ref: #9973
Closes #10017
2022-12-02 10:07:41 +01:00
Daniel Stenberg
3dfb6aeabb
runtests: only do CRLF replacements for hyper if it is HTTP
Closes #10016
2022-12-01 21:19:40 +01:00
Stefan Eissing
5c27f6c452
openssl: fix for BoringSSL BIO result interpretation mixups
Reported-by: Robin Marx
Fixes #10013
Closes #10015
2022-12-01 18:27:27 +01:00
Max Dymond
4c712a1b25 ci: Remove zuul fuzzing job as it's superseded by CIFuzz 2022-12-01 16:00:11 +00:00
Daniel Stenberg
2f34a7347f
runtests: do CRLF replacements per section only
The `crlf="yes"` attribute and "hyper mode" are now only applied on a
subset of dedicated sections: data, datacheck, stdout and protocol.

Updated test 2500 accordingly.

Also made test1 use crlf="yes" for <protocol>, mostly because it is
often used as a template test case. Going forward, using this attribute
we should be able to write test cases using linefeeds only and avoid
mixed line ending encodings.

Follow-up to ca15b7512e

Fixes #10009
Closes #10010
2022-12-01 15:04:23 +01:00
Stefan Eissing
27ec767ebd
gnutls: use common gnutls init and verify code for ngtcp2
Closes #10007
2022-12-01 10:57:36 +01:00
Baitinq on github
b8ffb02e84
aws_sigv4: fix typos in aws_sigv4.c
Closes #10008
2022-12-01 08:34:37 +01:00
Kenneth Myhra
610b96c6b3
curl.h: include <sys/select.h> on SerenityOS
Closes #10006
2022-11-30 15:37:44 +01:00
Daniel Stenberg
39bbd1cafb
openssl: prefix errors with '[lib]/[version]: '
To help users understand where this (cryptic) error message comes from.

Suggested-by: Philip Sanetra
Ref: #10002
Closes #10004
2022-11-30 14:05:57 +01:00
Stefan Eissing
ca15b7512e
tests: add HTTP/3 test case, custom location for proper nghttpx
- adding support for HTTP/3 test cases via a nghttpx server that is
  build with ngtcp2 and nghttp3.
- test2500 is the first test case, performing a simple GET.
- nghttpx is checked for support and the 'feature' nghttpx-h3
  is set accordingly. test2500 will only run, when supported.
- a specific nghttpx location can be given in the environment
  variable NGHTTPX or via the configure option
    --with-test-nghttpx=<path>

Extend NGHTTPX config to H2 tests as well

* use $ENV{NGHTTPX} and the configured default also in http2 server starts
* always provide the empty test/nghttpx.conf to nghttpx. as it defaults to
  reading /etc/nghttpx/nghttpx.conf otherwise.

Added nghttpx to CI ngtcp2 jobs to run h3 tests.

Closes #9031
2022-11-30 13:56:06 +01:00
Daniel Stenberg
0186ec41b1
RELEASE-NOTES: synced
Removed duplicate after contributors.sh fix: 9967c10b6d
2022-11-30 09:11:33 +01:00
Daniel Stenberg
9967c10b6d
scripts/contributors.sh: strip one OR MORE leading spaces
From names found credited in commit logs
2022-11-30 09:09:47 +01:00
Daniel Stenberg
4e2146554a
RELEASE-NOTES: synced 2022-11-30 08:59:43 +01:00
Daniel Stenberg
07203b0bb8
openssl/mbedtls: use %d for outputing port with failf (int)
Coverity CID 1517100

Also, remove some int typecasts in vtls.c for the port number

Closes #10001
2022-11-29 23:16:47 +01:00
Daniel Stenberg
f85e932b28
KNOWN_BUGS: remove "Multi perform hangs waiting for threaded resolver"
We now offer a way to avoid that hang, using CURLOPT_QUICK_EXIT.

Follow-up to 49798cac83 fixed via #9147

Closes #9999
2022-11-29 16:14:16 +01:00
Daniel Stenberg
862406c5e1
KNOWN_BUGS: remove "--interface for ipv6 binds to unusable IP address"
Since years back the "if2ip" function verifies that it binds to a local IPv6
address that uses the same scope as the remote address.

This is not a bug.

Fixes #686
Closes #9998
2022-11-29 16:13:10 +01:00
Daniel Stenberg
3075fdccdd
test1276: verify lib/optiontable.pl
Checks that it generates an output identical to the file.
2022-11-29 14:45:27 +01:00
Daniel Stenberg
ea557cfa27
lib/optiontable.pl: adapt to CURLOPTDEPRECATED()
Follow-up from 6967571bf2

Reported-by: Gisle Vanem

Fixes #9992
Closes #9993
2022-11-29 14:43:42 +01:00
Daniel Stenberg
f83610c89d
docs/INSTALL.md: list OSes and CPUs quoted
to make them skip spellcheck. Also added a new CPU.

Follow-up to 4506cbf7f2

Closes #9997
2022-11-29 10:28:31 +01:00
Ikko Ashimine
ae99ac6ffb vtls: fix typo in vtls_int.h
paramter -> parameter

Closes: #9996
Reviewed-by: Daniel Gustafsson <daniel@yesql.se>
2022-11-28 19:51:22 +01:00
Daniel Stenberg
42aa4e1a70
curl-openssl.m4: do not add $prefix/include/openssl to CPPFLAGS
As OpenSSL's include files are all included using <openssl/*.h> in curl
source code, we just risk that existing openssl files will "shadow"
include files without path if that path is provided.

Fixes #9989
Closes #9988
2022-11-28 17:27:27 +01:00
Daniel Stenberg
4506cbf7f2
INSTALL: update operating systems and CPU archs
Update after recent runs on Twitter/Mastodon and my blog

Closes #9994
2022-11-28 17:26:40 +01:00
Stefan Eissing
55807e6c05
tls: backends use connection filters for IO, enabling HTTPS-proxy
- OpenSSL (and compatible)
 - BearSSL
 - gnutls
 - mbedtls
 - rustls
 - schannel
 - secure-transport
 - wolfSSL (v5.0.0 and newer)

 This leaves only the following without HTTPS-proxy support:
 - gskit
 - nss
 - wolfSSL (versions earlier than v5.0.0)

Closes #9962
2022-11-28 13:56:23 +01:00
Daniel Stenberg
dbd74baf78
include/curl/curl.h: bump the deprecated requirements to gcc 6.1
Reported-by: Michael Kaufmann
Fixes #9917
Closes #9987
2022-11-28 13:13:53 +01:00
Patrick Monnerat
eb559c8056
mime: relax easy/mime structures binding
Deprecation and removal of codeset conversion support from the library
have released the strict need for an early binding of mime structures to
an easy handle (https://github.com/curl/curl/commit/2610142).

This constraint currently forces to create the handle before the mime
structure and the latter cannot be attached to another handle once
created (see https://curl.se/mail/lib-2022-08/0027.html).

This commit removes the handle pointers from the mime structures
allowing more flexibility on their use.

When an easy handle is duplicated, bound mime structures must however
still be duplicated too as their components hold send-time dynamic
information.

Closes #9927
2022-11-28 09:09:54 +01:00
fractal-access
383fb29da1
test416: verify growing FTP file support
Added setting: RETRSIZE [size] in the <servercmd> section. When set this
will cause the test FTP server to return the size set (rather than the
actual size) in the acknowledgement from a RETR request.

Closes #9772
2022-11-26 17:06:28 +01:00
fractal-access
76b3f5f2cf
ftp: support growing files with CURLOPT_IGNORE_CONTENT_LENGTH
When using the option CURLOPT_IGNORE_CONTENT_LENGTH (set.ignorecl in
code) to support growing files in FTP, the code should ignore the
initial size it gets from the server as this will not be the final size
of the file. This is done in ftp_state_quote() to prevent a size request
being issued in the initial sequence. However, in a later call to
ftp_state_get_resp() the code attempts to get the size of the content
again if it doesn't already have it, by parsing the response from the
RETR request. This fix prevents this parsing of the response to get the
size when the set.ignorecl option is set. This should maintain the size
value as -1, unknown, in this situation.

Closes #9772
2022-11-26 17:02:54 +01:00
Stefan Eissing
f18956d0ca
cfilter: re-add conn as parameter to cfilter setup methods
- `Curl_ssl_get_config()` now returns the first config if no SSL proxy
  filter is active

- socket filter starts connection only on first invocation of its
  connect method

Fixes #9982
Closes #9983
2022-11-26 16:56:46 +01:00
Daniel Stenberg
3e33681eaf
KNOWN_BUGS: remove five FTP related issues
- "FTP with CONNECT and slow server"

I believe this is not a problem these days.

- "FTP with NULs in URL parts"

The FTP protocol does not support them properly anyway.

- remove "FTP and empty path parts in the URL"

I don't think this has ever been reported as a real problem but was only
a hypothetical one.

- "Premature transfer end but healthy control channel"

This is not a bug, this is an optimization that *could* be performed but is
not an actual problem.

- "FTP without or slow 220 response"

Instead add to the documentation of the connect timeout that the
connection is considered complete at TCP/TLS/QUIC layer.

Closes #9979
2022-11-26 12:33:58 +01:00
Stefan Eissing
f70297187f
tests: add authorityInfoAccess to generated certs
Generate stunnel.pem as well

Closes #9980
2022-11-26 00:11:57 +01:00
Daniel Stenberg
280cbeee27
runtests: --no-debuginfod now disables DEBUGINFOD_URLS
Prior to this change, DEBUGINFOD_URLS was always disabled by runtests
due to a report of it slowing down tests. However, some setups need it
to fetch debug symbols, and if it is disabled on those systems then curl
tests with valgrind will fail.

Reported-by: Mark Gaiser

Ref: #8805
Closes #9950
2022-11-25 09:33:52 +01:00
Casey Bodley
fcf8f68546
test/aws_sigv4: test cases for content-sha256
1956 adds the sha256 value corresponding to an empty buffer
1957 adds an arbitrary value and confirms that the signature differs from 1956
1958 adds whitespace to 1957 and confirms that the signature matches 1957
1959 adds a value longer than 'char sha_hex[65]' in Curl_output_aws_sigv4()

Signed-off-by: Casey Bodley <cbodley@redhat.com>

Closes #9804
2022-11-25 09:24:43 +01:00
Casey Bodley
7f8e6da6dc
aws_sigv4: consult x-%s-content-sha256 for payload hash
`Curl_output_aws_sigv4()` doesn't always have the whole payload in
memory to generate a real payload hash. this commit allows the user to
pass in a header like `x-amz-content-sha256` to provide their desired
payload hash

some services like s3 require this header, and may support other values
like s3's `UNSIGNED-PAYLOAD` and `STREAMING-AWS4-HMAC-SHA256-PAYLOAD`
with special semantics. servers use this header's value as the payload
hash during signature validation, so it must match what the client uses
to generate the signature

CURLOPT_AWS_SIGV4.3 now describes the content-sha256 interaction

Signed-off-by: Casey Bodley <cbodley@redhat.com>

Closes #9804
2022-11-25 09:24:07 +01:00
Philip H
4c61a8e8f4
GHA: NSS use clang instead of clang-9
Closes #9978
2022-11-25 08:59:22 +01:00
Daniel Stenberg
e3302eb5bb
RELEASE-NOTES: synced 2022-11-25 08:40:19 +01:00
Daniel Stenberg
975efec61b
tool_operate: override the numeric locale and set "C" by force
Makes curl always use dot as decimal separator for options,
independently of what the locale says. Makes scripts and command lines
portable.

Updated docs accordingly.

Reported-by: Daniel Faust

Fixes #9969
Closes #9972
2022-11-25 08:32:23 +01:00
Daniel Stenberg
cde8f28489
test1662: verify formpost, 301 redirect, no rewind possible
Reproduces #9735 and verifies the subsequent fix. The original issue
uses a pipe that cannot be rewound, but this test case instead sets a
callback without rewind ability to get roughly the same properties but
being a much more portable test.
2022-11-25 08:30:29 +01:00
Daniel Stenberg
1b3973180b
lib: rewind BEFORE request instead of AFTER previous
This makes a big difference for cases when the rewind is not actually
necessary to perofm (for example HTTP response code 301 converts to GET)
and therefore the rewind can be avoided. In particular for situations
when that rewind fails, for example when reading from a pipe or similar.

Reported-by: Ali Utku Selen

Fixes #9735
Closes #9958
2022-11-25 08:30:26 +01:00
Daniel Stenberg
9ecebb5d00
vtls: repair build with disabled proxy
Closes #9974
2022-11-24 09:04:14 +01:00
Daniel Gustafsson
8b9c5bef9b packaging: remove traces of deleted files
Commit a8861b6cc removed packages/DOS but left a few traces of it
which broke the distcheck CI. Remove all traces.

Closes: #9971
Reviewed-by: Daniel Stenberg <daniel@haxx.se>
2022-11-23 12:12:59 +01:00
Daniel Gustafsson
3ae63a121d openssl: silence compiler warning when not using IPv6
In non-IPv6 builds the conn parameter is unused, and compilers which
run with "-Werror=unused-parameter" (or similar) warnings turned on
fails to build. Below is an excerpt from a CI job:

  vtls/openssl.c: In function ‘Curl_ossl_verifyhost’:
  vtls/openssl.c:2016:75: error: unused parameter ‘conn’ [-Werror=unused-parameter]
   2016 | CURLcode Curl_ossl_verifyhost(struct Curl_easy *data, struct connectdata *conn,
        |                                                       ~~~~~~~~~~~~~~~~~~~~^~~~

Closes: #9970
Reviewed-by: Daniel Stenberg <daniel@haxx.se>
2022-11-23 12:10:30 +01:00
Daniel Gustafsson
2d9fee4504 netware: remove leftover traces
Commit 3b16575ae9 removed support for
building on Novell Netware, but a few leftover traces remained. This
removes the last bits.

Closes: #9966
Reviewed-by: Daniel Stenberg <daniel@haxx.se>
2022-11-23 11:56:52 +01:00
Ryan Schmidt
e007ace30d
curl_endian: remove Curl_write64_le from header
The actual function was already removed in 4331c6dc.

See #7280
Closes #9968
2022-11-23 09:07:12 +01:00
Daniel Stenberg
66ca8ace1a
docs: add more "SEE ALSO" links to CA related pages
Closes #9959
2022-11-22 14:43:38 +01:00
Daniel Stenberg
c1cdc08cbb
examples: update descriptions
Make them not say "this is an example showing..." and instead just say
what the example shows.

Closes #9960
2022-11-22 14:41:15 +01:00
Stefan Eissing
af22c2a546
vtls: localization of state data in filters
- almost all backend calls pass the Curl_cfilter intance instead of
   connectdata+sockindex
 - ssl_connect_data is remove from struct connectdata and made internal
   to vtls
 - ssl_connect_data is allocated in the added filter, kept at cf->ctx

 - added function to let a ssl filter access its ssl_primary_config and
   ssl_config_data this selects the propert subfields in conn and data,
   for filters added as plain or proxy
 - adjusted all backends to use the changed api
 - adjusted all backends to access config data via the exposed
   functions, no longer using conn or data directly

cfilter renames for clear purpose:

 - methods `Curl_conn_*(data, conn, sockindex)` work on the complete
   filter chain at `sockindex` and connection `conn`.
 - methods `Curl_cf_*(cf, ...)` work on a specific Curl_cfilter
   instance.
 - methods `Curl_conn_cf()` work on/with filter instances at a
   connection.
 - rebased and resolved some naming conflicts
 - hostname validation (und session lookup) on SECONDARY use the same
   name as on FIRST (again).

new debug macros and removing connectdata from function signatures where not
needed.

adapting schannel for new Curl_read_plain paramter.

Closes #9919
2022-11-22 14:25:50 +01:00