Commit Graph

63 Commits

Author SHA1 Message Date
max.mehl
ad9bc5976d
copyright: make repository REUSE compliant
Add licensing and copyright information for all files in this repository. This
either happens in the file itself as a comment header or in the file
`.reuse/dep5`.

This commit also adds a Github workflow to check pull requests and adapts
copyright.pl to the changes.

Closes #8869
2022-06-13 09:13:00 +02:00
Daniel Gustafsson
e70928b091 doh: remove UNITTEST macro definition
The UNITTEST macro is defined by curl_setup.h so there is no use in
carry a local copy of the logic.

Closes: #8902
Reviewed-by: Daniel Stenberg <daniel@haxx.se>
2022-05-24 12:46:22 +02:00
Daniel Stenberg
8d86718f3f
opts: deprecate RANDOM_FILE and EGDSOCKET
These two options were only ever used for the OpenSSL backend for
versions before 1.1.0. They were never used for other backends and they
are not used with recent OpenSSL versions. They were never used much by
applications.

The defines RANDOM_FILE and EGD_SOCKET can still be set at build-time
for ancient EOL OpenSSL versions.

Closes #8670
2022-05-17 11:05:54 +02:00
Daniel Stenberg
b716b5aa7e
lib: #ifdef on USE_HTTP2 better
... as nghttp2 might not be the library that provides HTTP/2 support.

Closes #8661
2022-04-01 13:45:52 +02:00
HenrikHolst
b807219292
misc: reduce strlen() calls with Curl_dyn_add()
Use STRCONST() to switch from Curl_dyn_add() to Curl_dyn_addn() for
string literals.

Closes #8398
2022-02-09 13:52:47 +01:00
Daniel Stenberg
91e3996aa7
urldata: CONN_IS_PROXIED replaces bits.close when proxy can be disabled
To remove run-time checks for such builds.

Closes #8350
2022-01-30 12:01:25 +01:00
Daniel Gustafsson
f06280eb53 doh: remove experimental code for DoH with GET
The code for sending DoH requests with GET was never enabled in a way
such that it could be used or tested. As there haven't been requests
for this feature, and since it at this is effectively dead, remove it
and favor reimplementing the feature in case anyone is interested.

Closes #7870
Reviewed-by: Daniel Stenberg <daniel@haxx.se>
2021-10-18 13:25:57 +02:00
Josh Soref
de1004eb0f
cleanup: spell DoH with a lowercase o
Signed-off-by: Josh Soref <jsoref@users.noreply.github.com>

Closes #7413
2021-07-16 23:53:13 +02:00
Daniel Stenberg
e7416cfd2b
infof: remove newline from format strings, always append it
- the data needs to be "line-based" anyway since it's also passed to the
  debug callback/application

- it makes infof() work like failf() and consistency is good

- there's an assert that triggers on newlines in the format string

- Also removes a few instances of "..."

- Removes the code that would append "..." to the end of the data *iff*
  it was truncated in infof()

Closes #7357
2021-07-07 22:54:01 +02:00
Daniel Stenberg
6a40e6f8be
doh: (void)-prefix call to curl_easy_setopt 2021-07-01 00:06:27 +02:00
zhangxiuhua
23eef2394c
doh: fix wrong DEBUGASSERT for doh private_data
Closes #7227
2021-06-10 16:13:13 +02:00
Lucas Clemente Vella
84d2839740
CURLOPT_IPRESOLVE: preventing wrong IP version from being used
In some situations, it was possible that a transfer was setup to
use an specific IP version, but due do DNS caching or connection
reuse, it ended up using a different IP version from requested.

This commit changes the effect of CURLOPT_IPRESOLVE from simply
restricting address resolution to preventing the wrong connection
type being used, when choosing a connection from the pool, and
to restricting what addresses could be used when establishing
a new connection.

It is important that all addresses versions are resolved, even if
not used in that transfer in particular, because the result is
cached, and could be useful for a different transfer with a
different CURLOPT_IPRESOLVE setting.

Closes #6853
2021-05-20 16:58:31 +02:00
Gilles Vollant
77fc3859b2 SSL: support in-memory CA certs for some backends
- New options CURLOPT_CAINFO_BLOB and CURLOPT_PROXY_CAINFO_BLOB to
  specify in-memory PEM certificates for OpenSSL, Schannel (Windows)
  and Secure Transport (Apple) SSL backends.

Prior to this change PEM certificates could only be imported from a file
and not from memory.

Co-authored-by: moparisthebest@users.noreply.github.com

Ref: https://github.com/curl/curl/pull/4679
Ref: https://github.com/curl/curl/pull/5677
Ref: https://github.com/curl/curl/pull/6109

Closes https://github.com/curl/curl/pull/6662
2021-05-05 02:29:16 -04:00
Jay Satiro
54e7475016 schannel: Disable auto credentials; add an option to enable it
- Disable auto credentials by default. This is a breaking change
  for clients that are using it, wittingly or not.

- New libcurl ssl option value CURLSSLOPT_AUTO_CLIENT_CERT tells libcurl
  to automatically locate and use a client certificate for
  authentication, when requested by the server.

- New curl tool options --ssl-auto-client-cert and
  --proxy-ssl-auto-client-cert map to CURLSSLOPT_AUTO_CLIENT_CERT.

This option is only supported for Schannel (the native Windows SSL
library). Prior to this change Schannel would, with no notification to
the client, attempt to locate a client certificate and send it to the
server, when requested by the server. Since the server can request any
certificate that supports client authentication in the OS certificate
store it could be a privacy violation and unexpected.

Fixes https://github.com/curl/curl/issues/2262
Reported-by: Jeroen Ooms
Assisted-by: Wes Hinsley
Assisted-by: Rich FitzJohn

Ref: https://curl.se/mail/lib-2021-02/0066.html
Reported-by: Morten Minde Neergaard

Closes https://github.com/curl/curl/pull/6673
2021-04-22 16:53:37 -04:00
Jay Satiro
8a4ef73c8f docs: Explain DOH transfers inherit some SSL settings
- Document in DOH that some SSL settings are inherited but DOH hostname
  and peer verification are not and are controlled separately.

- Document that CURLOPT_SSL_CTX_FUNCTION is inherited by DOH handles but
  we're considering changing behavior to no longer inherit it. Request
  feedback.

Closes https://github.com/curl/curl/pull/6688
2021-03-17 18:12:45 -04:00
Jay Satiro
1ba0d4bdb3 doh: Inherit CURLOPT_STDERR from user's easy handle
Prior to this change if the user set their easy handle's error stream
to something other than stderr it was not inherited by the doh handles,
which meant that they would still write to the default standard error
stream (stderr) for verbose output.

Bug: https://github.com/curl/curl/issues/6605
Reported-by: arvids-kokins-bidstack@users.noreply.github.com

Closes https://github.com/curl/curl/pull/6661
2021-03-02 16:59:28 -05:00
Daniel Stenberg
70472a44de
urldata: remove the _ORIG suffix from string names
It doesn't provide any useful info but only makes the names longer.

Closes #6624
2021-02-19 08:07:19 +01:00
Jay Satiro
e68ee39f3d doh: Fix sharing user's resolve list with DOH handles
- Share the shared object from the user's easy handle with the DOH
  handles.

Prior to this change if the user had set a shared object with shared
cached DNS (CURL_LOCK_DATA_DNS) for their easy handle then that wasn't
used by any associated DOH handles, since they used the multi's default
hostcache.

This change means all the handles now use the same hostcache, which is
either the shared hostcache from the user created shared object if it
exists or if not then the multi's default hostcache.

Reported-by: Manuj Bhatia

Fixes https://github.com/curl/curl/issues/6589
Closes https://github.com/curl/curl/pull/6607
2021-02-15 16:14:34 -05:00
Jay Satiro
53022e1893 doh: add options to disable ssl verification
- New libcurl options CURLOPT_DOH_SSL_VERIFYHOST,
  CURLOPT_DOH_SSL_VERIFYPEER and CURLOPT_DOH_SSL_VERIFYSTATUS do the
  same as their respective counterparts.

- New curl tool options --doh-insecure and --doh-cert-status do the same
  as their respective counterparts.

Prior to this change DOH SSL certificate verification settings for
verifyhost and verifypeer were supposed to be inherited respectively
from CURLOPT_SSL_VERIFYHOST and CURLOPT_SSL_VERIFYPEER, but due to a bug
were not. As a result DOH verification remained at the default, ie
enabled, and it was not possible to disable. This commit changes
behavior so that the DOH verification settings are independent and not
inherited.

Ref: https://github.com/curl/curl/pull/4579#issuecomment-554723676

Fixes https://github.com/curl/curl/issues/4578
Closes https://github.com/curl/curl/pull/6597
2021-02-14 18:20:48 -05:00
Daniel Stenberg
8335c6417e
hostip: remove conn->data from resolver functions
This also moves the 'async' struct from the connectdata struct into the
Curl_easy struct, which seems like a better home for it.

Closes #6497
2021-01-22 08:25:09 +01:00
Daniel Stenberg
b97a60f5f4
doh: make Curl_doh_is_resolved survive a NULL pointer
... if Curl_doh() returned a NULL, this function gets called anyway as
in a asynch procedure. Then the doh struct pointer is NULL and signifies
an OOM situation.

Follow-up to 6246a1d8c6
2021-01-21 23:31:19 +01:00
Daniel Stenberg
6246a1d8c6
doh: allocate state struct on demand
... instead of having it static within the Curl_easy struct. This takes
away 1176 bytes (18%) from the Curl_easy struct that aren't used very
often and instead makes the code allocate it when needed.

Closes #6492
2021-01-20 09:02:24 +01:00
Daniel Stenberg
476e1339ef
doh: remove Curl_ prefix from static functions 2021-01-14 08:09:09 +01:00
Emil Engler
725ec470e2
language: s/behaviour/behavior/g
We currently use both spellings the british "behaviour" and the american
"behavior". However "behavior" is more used in the project so I think
it's worth dropping the british name.

Closes #6395
2021-01-02 23:35:59 +01:00
Daniel Stenberg
aba01da639
failf: remove newline from formatting strings
... as failf adds one itself.

Also: add an assert() to failf() that triggers on a newline in the
format string!

Closes #6365
2020-12-25 23:42:16 +01:00
Daniel Stenberg
4d2f800677
curl.se: new home
Closes #6172
2020-11-04 23:59:47 +01:00
Michael Baentsch
ede125b7b7
tls: add CURLOPT_SSL_EC_CURVES and --curves
Closes #5892
2020-08-30 17:24:04 +02:00
Emil Engler
a6a17662f2
doh: add error message for DOH_DNS_NAME_TOO_LONG
When this error code was introduced in b6a53fff6c, it was
forgotten to be added in the errors array and doh_strerror function.

Closes #5863
2020-08-26 22:51:29 +02:00
Marcel Raad
d746ff1107
doh: remove redundant cast
Closes https://github.com/curl/curl/pull/5704
2020-07-21 20:00:29 +02:00
Daniel Stenberg
54d3769761
Curl_addrinfo: use one malloc instead of three
To reduce the amount of allocations needed for creating a Curl_addrinfo
struct, make a single larger malloc instead of three separate smaller
ones.

Closes #5533
2020-06-08 16:10:53 +02:00
Daniel Stenberg
67d2802dea
hostip: on macOS avoid DoH when given a numerical IP address
When USE_RESOLVE_ON_IPS is set (defined on macOS), it means that
numerical IP addresses still need to get "resolved" - but not with DoH.

Reported-by: Viktor Szakats
Fixes #5454
Closes #5459
2020-05-26 17:37:39 +02:00
Daniel Stenberg
8df455479f
source cleanup: remove all custom typedef structs
- Stick to a single unified way to use structs
 - Make checksrc complain on 'typedef struct {'
 - Allow them in tests, public headers and examples

 - Let MD4_CTX, MD5_CTX, and SHA256_CTX typedefs remain as they actually
   typedef different types/structs depending on build conditions.

Closes #5338
2020-05-15 08:54:42 +02:00
Daniel Stenberg
ed35d6590e
dynbuf: introduce internal generic dynamic buffer functions
A common set of functions instead of many separate implementations for
creating buffers that can grow when appending data to them. Existing
functionality has been ported over.

In my early basic testing, the total number of allocations seem at
roughly the same amount as before, possibly a few less.

See docs/DYNBUF.md for a description of the API.

Closes #5300
2020-05-04 10:40:39 +02:00
Rikard Falkeborn
fc0e29dd57
doh: Constify some input pointers
Closes #5306
2020-04-28 07:56:34 +02:00
Johannes Schindelin
5450428491 schannel: add "best effort" revocation check option
- Implement new option CURLSSLOPT_REVOKE_BEST_EFFORT and
  --ssl-revoke-best-effort to allow a "best effort" revocation check.

A best effort revocation check ignores errors that the revocation check
was unable to take place. The reasoning is described in detail below and
discussed further in the PR.

---

When running e.g. with Fiddler, the schannel backend fails with an
unhelpful error message:

	Unknown error (0x80092012) - The revocation function was unable
	to check revocation for the certificate.

Sadly, many enterprise users who are stuck behind MITM proxies suffer
the very same problem.

This has been discussed in plenty of issues:
https://github.com/curl/curl/issues/3727,
https://github.com/curl/curl/issues/264, for example.

In the latter, a Microsoft Edge developer even made the case that the
common behavior is to ignore issues when a certificate has no recorded
distribution point for revocation lists, or when the server is offline.
This is also known as "best effort" strategy and addresses the Fiddler
issue.

Unfortunately, this strategy was not chosen as the default for schannel
(and is therefore a backend-specific behavior: OpenSSL seems to happily
ignore the offline servers and missing distribution points).

To maintain backward-compatibility, we therefore add a new flag
(`CURLSSLOPT_REVOKE_BEST_EFFORT`) and a new option
(`--ssl-revoke-best-effort`) to select the new behavior.

Due to the many related issues Git for Windows and GitHub Desktop, the
plan is to make this behavior the default in these software packages.

The test 2070 was added to verify this behavior, adapted from 310.

Based-on-work-by: georgeok <giorgos.n.oikonomou@gmail.com>
Co-authored-by: Markus Olsson <j.markus.olsson@gmail.com>
Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>

Closes https://github.com/curl/curl/pull/4981
2020-03-18 03:23:39 -04:00
Pedro Monreal
4b6fd29f1a cleanup: fix typos and wording in docs and comments
Closes #4869
Reviewed-by: Emil Engler and Daniel Gustafsson
2020-02-02 18:43:01 +01:00
Daniel Stenberg
4940bb8568
doh: make it behave when built without proxy support
Reported-by: Marcel Raad
Bug: https://github.com/curl/curl/pull/4692#issuecomment-564115734

Closes #4704
2019-12-12 09:26:08 +01:00
Jay Satiro
9c1806ae46 build: Disable Visual Studio warning "conditional expression is constant"
- Disable warning C4127 "conditional expression is constant" globally
  in curl_setup.h for when building with Microsoft's compiler.

This mainly affects building with the Visual Studio project files found
in the projects dir.

Prior to this change the cmake and winbuild build systems already
disabled 4127 globally for when building with Microsoft's compiler.
Also, 4127 was already disabled for all build systems in the limited
circumstance of the WHILE_FALSE macro which disabled the warning
specifically for while(0). This commit removes the WHILE_FALSE macro and
all other cruft in favor of disabling globally in curl_setup.

Background:

We have various macros that cause 0 or 1 to be evaluated, which would
cause warning C4127 in Visual Studio. For example this causes it:

    #define Curl_resolver_asynch() 1

Full behavior is not clearly defined and inconsistent across versions.
However it is documented that since VS 2015 Update 3 Microsoft has
addressed this somewhat but not entirely, not warning on while(true) for
example.

Prior to this change some C4127 warnings occurred when I built with
Visual Studio using the generated projects in the projects dir.

Closes https://github.com/curl/curl/pull/4658
2019-12-01 19:01:02 -05:00
Niall
8acfad38ca
doh: use dedicated probe slots
... to easier allow additional DNS transactions.

Closes #4629
2019-11-28 12:49:44 +01:00
Niall
b6a53fff6c
doh: improced both encoding and decoding
Improved estimation of expected_len and updated related comments;
increased strictness of QNAME-encoding, adding error detection for empty
labels and names longer than the overall limit; avoided treating DNAME
as unexpected;

updated unit test 1655 with more thorough set of proofs and tests

Closes #4598
2019-11-16 16:15:03 +01:00
Daniel Stenberg
dcd7e37c3a
url: make Curl_close() NULLify the pointer too
This is the common pattern used in the code and by a unified approach we
avoid mistakes.

Closes #4534
2019-10-30 09:36:21 +01:00
Paul Dreik
a5bf6a36c5
doh: allow only http and https in debug mode
Otherwise curl may be told to use for instance pop3 to
communicate with the doh server, which most likely
is not what you want.

Found through fuzzing.

Closes #4406
2019-09-23 22:38:23 +02:00
Paul Dreik
bb74201804
doh: return early if there is no time left
Closes #4406
2019-09-23 22:38:15 +02:00
Niall O'Reilly
0d59addff6
doh: avoid truncating DNS QTYPE to lower octet
Closes #4381
2019-09-19 22:50:38 +02:00
Daniel Stenberg
9bc44ff64d
doh: clean up dangling DOH handles and memory on easy close
If you set the same URL for target as for DoH (and it isn't a DoH
server), like "https://example.com" in both, the easy handles used for
the DoH requests could be left "dangling" and end up not getting freed.

Reported-by: Paul Dreik
Closes #4366
2019-09-16 17:31:56 +02:00
Paul Dreik
dda418266c
doh: fix undefined behaviour and open up for gcc and clang optimization
The undefined behaviour is annoying when running fuzzing with
sanitizers. The codegen is the same, but the meaning is now not up for
dispute. See https://cppinsights.io/s/516a2ff4

By incrementing the pointer first, both gcc and clang recognize this as
a bswap and optimizes it to a single instruction.  See
https://godbolt.org/z/994Zpx

Closes #4350
2019-09-15 23:27:45 +02:00
Paul Dreik
b766602729
doh: fix (harmless) buffer overrun
Added unit test case 1655 to verify.
Close #4352

the code correctly finds the flaws in the old code,
if one temporarily restores doh.c to the old version.
2019-09-15 23:25:24 +02:00
Marcel Raad
10db3ef21e
lib: reduce variable scopes
Fixes Codacy/CppCheck warnings.

Closes https://github.com/curl/curl/pull/3872
2019-05-20 08:51:11 +02:00
Daniel Stenberg
ae4adae954
doh: CURL_DISABLE_DOH 2019-05-13 08:17:09 +02:00
Jay Satiro
9e6af114ea doh: inherit some SSL options from user's easy handle
- Inherit SSL options for the doh handle but not SSL client certs,
  SSL ALPN/NPN, SSL engine, SSL version, SSL issuer cert,
  SSL pinned public key, SSL ciphers, SSL id cache setting,
  SSL kerberos or SSL gss-api settings.

- Fix inheritance of verbose setting.

- Inherit NOSIGNAL.

There is no way for the user to set options for the doh (DNS-over-HTTPS)
handles and instead we inherit some options from the user's easy handle.

My thinking for the SSL options not inherited is they are most likely
not intended by the user for the DOH transfer. I did inherit insecure
because I think that should still be in control of the user.

Prior to this change doh did not work for me because CAINFO was not
inherited. Also verbose was set always which AFAICT was a bug (#3660).

Fixes https://github.com/curl/curl/issues/3660
Closes https://github.com/curl/curl/pull/3661
2019-03-11 01:50:12 -04:00