Commit Graph

77 Commits

Author SHA1 Message Date
Daniel Stenberg
030adbceeb
conncache: remove name arg from Curl_conncache_find_bundle
To simplify, and also since the returned name is not the full actual
name used for the check. The port number and zone id is also involved,
so just showing the name is misleading.

Closes #8750
2022-04-25 16:25:42 +02:00
Daniel Stenberg
058f98dc3f
conncache: include the zone id in the "bundle" hashkey
Make connections to two separate IPv6 zone ids create separate
connections.

Reported-by: Harry Sintonen
Bug: https://curl.se/docs/CVE-2022-27775.html
Closes #8747
2022-04-25 11:48:00 +02:00
Daniel Stenberg
d4cf98bbd1
connecache: remove duplicate connc->closure_handle check
The superfluous extra check could cause analyzer false positives
and doesn't serve any purpose.

Closes #8676
2022-04-06 08:32:04 +02:00
Daniel Stenberg
79cca04da8
conncache: make conncache_add_bundle return the pointer
Simplifies the logic a little and avoids a ternary operator.

Ref: #8346
Closes #8349
2022-01-30 11:59:43 +01:00
Daniel Stenberg
e74a6b7b72
url: make Curl_disconnect return void
1. The function would only ever return CURLE_OK anyway
 2. Only one caller actually used the return code
 3. Most callers did (void)Curl_disconnect()

Closes #8303
2022-01-20 09:07:30 +01:00
Daniel Stenberg
254f7bd78a
hash: lazy-alloc the table in Curl_hash_add()
This makes Curl_hash_init() infallible which saves error paths.

Closes #8132
2021-12-10 23:16:43 +01:00
Daniel Stenberg
e7416cfd2b
infof: remove newline from format strings, always append it
- the data needs to be "line-based" anyway since it's also passed to the
  debug callback/application

- it makes infof() work like failf() and consistency is good

- there's an assert that triggers on newlines in the format string

- Also removes a few instances of "..."

- Removes the code that would append "..." to the end of the data *iff*
  it was truncated in infof()

Closes #7357
2021-07-07 22:54:01 +02:00
Daniel Stenberg
dddad339e8
conncache: lowercase the hash key for better match
As host names are case insensitive, the use of case sensitive hashing
caused unnecesary cache misses and therefore lost performance. This
lowercases the hash key.

Reported-by: Harry Sintonen
Fixes #7159
Closes #7161
2021-06-01 17:55:35 +02:00
Daniel Stenberg
cfff12a0b3
lib: remove 'conn->data' completely
The Curl_easy pointer struct entry in connectdata is now gone. Just
before commit 215db086e0 landed on January 8, 2021 there were 919
references to conn->data.

Closes #6608
2021-02-16 10:48:57 +01:00
Daniel Stenberg
a304051620
lib: more conn->data cleanups
Closes #6479
2021-01-19 09:14:51 +01:00
Daniel Stenberg
215db086e0
lib: pass in 'struct Curl_easy *' to most functions
... in most cases instead of 'struct connectdata *' but in some cases in
addition to.

- We mostly operate on transfers and not connections.

- We need the transfer handle to log, store data and more. Everything in
  libcurl is driven by a transfer (the CURL * in the public API).

- This work clarifies and separates the transfers from the connections
  better.

- We should avoid "conn->data". Since individual connections can be used
  by many transfers when multiplexing, making sure that conn->data
  points to the current and correct transfer at all times is difficult
  and has been notoriously error-prone over the years. The goal is to
  ultimately remove the conn->data pointer for this reason.

Closes #6425
2021-01-17 23:56:09 +01:00
Daniel Stenberg
4d2f800677
curl.se: new home
Closes #6172
2020-11-04 23:59:47 +01:00
Daniel Stenberg
9e90ff0839
hash: make it 'struct Curl_hash'
As internal global names should use captical C.

Closes #5906
2020-09-02 22:41:58 +02:00
Daniel Stenberg
9b3f888a00
llist: make it "struct Curl_llist"
As internal global names should use captical C.

Closes #5906
2020-09-02 22:41:58 +02:00
Daniel Stenberg
8346c90b78
conncache: download buffer needs +1 size for trailing zero
Follow-up to c4e6968127
Detected by OSS-Fuzz: https://oss-fuzz.com/testcase-detail/5727799779524608
2020-05-31 17:45:57 +02:00
Daniel Stenberg
f3d501dc67
build: disable more code/data when built without proxy support
Added build to travis to verify

Closes #5466
2020-05-30 23:18:16 +02:00
Daniel Stenberg
c4e6968127
url: alloc the download buffer at transfer start
... and free it as soon as the transfer is done. It removes the extra
alloc when a new size is set with setopt() and reduces memory for unused
easy handles.

In addition: the closure_handle now doesn't use an allocated buffer at
all but the smallest supported size as a stack based one.

Closes #5472
2020-05-30 23:14:33 +02:00
Daniel Stenberg
c069027139
conncache: various concept cleanups
More connection cache accesses are protected by locks.

CONNCACHE_* is a beter prefix for the connection cache lock macros.

Curl_attach_connnection: now called as soon as there's a connection
struct available and before the connection is added to the connection
cache.

Curl_disconnect: now assumes that the connection is already removed from
the connection cache.

Ref: #4915
Closes #5009
2020-04-30 14:27:54 +02:00
Daniel Stenberg
7745000338
conncache: removed unused Curl_conncache_bundle_size() 2020-01-24 10:29:06 +01:00
Daniel Stenberg
1d5c427d7f
conncache: CONNECT_ONLY connections assumed always in-use
This makes them never to be considered "the oldest" to be discarded when
reaching the connection cache limit. The reasoning here is that
CONNECT_ONLY is primarily used in combination with using the
connection's socket post connect and since that is used outside of
curl's knowledge we must assume that it is in use until explicitly
closed.

Reported-by: Pavel Pavlov
Reported-by: Pavel Löbl
Fixes #4426
Fixes #4369
Closes #4696
2019-12-11 09:25:56 +01:00
Daniel Stenberg
ee263de7a3
conncache: fix multi-thread use of shared connection cache
It could accidentally let the connection get used by more than one
thread, leading to double-free and more.

Reported-by: Christopher Reid
Fixes #4544
Closes #4557
2019-12-09 15:30:09 +01:00
Daniel Stenberg
dcd7e37c3a
url: make Curl_close() NULLify the pointer too
This is the common pattern used in the code and by a unified approach we
avoid mistakes.

Closes #4534
2019-10-30 09:36:21 +01:00
Steve Holme
bdf6d8af3e
conncache: Remove the DEBUGASSERT on length check
We trust the calling code as this is an internal function.

Closes #3962
2019-05-29 22:07:43 +01:00
Daniel Stenberg
a6183ab23a
url: default conn->port to the same as conn->remote_port
... so that it has a sensible value when ConnectionExists() is called which
needs it set to differentiate host "bundles" correctly on port number!

Also, make conncache:hashkey() use correct port for bundles that are proxy vs
host connections.

Probably a regression from 7.62.0

Reported-by: Tom van der Woerdt
Fixes #3956
Closes #3957
2019-05-28 23:00:42 +02:00
Daniel Stenberg
2f11fbf208
conncache: make "bundles" per host name when doing proxy tunnels
Only HTTP proxy use where multiple host names can be used over the same
connection should use the proxy host name for bundles.

Reported-by: Tom van der Woerdt
Fixes #3951
Closes #3955
2019-05-28 16:23:59 +02:00
Daniel Stenberg
e649432e72
CURLOPT_MAXAGE_CONN: set the maximum allowed age for conn reuse
... and disconnect too old ones instead of trying to reuse.

Default max age is set to 118 seconds.

Ref: #3722
Closes #3782
2019-04-21 23:06:23 +02:00
Daniel Stenberg
e3a53e3efb
conncache: use conn->data to know if a transfer owns it
- make sure an already "owned" connection isn't returned unless
  multiplexed.

- clear ->data when returning the connection to the cache again

Regression since 7.62.0 (probably in commit 1b76c38904)

Bug: https://curl.haxx.se/mail/lib-2019-03/0064.html

Closes #3686
2019-03-17 18:07:20 +01:00
Daniel Stenberg
05b100aee2
cleanup: make local functions static
urlapi: turn three local-only functions into statics

conncache: make conncache_find_first_connection static

multi: make detach_connnection static

connect: make getaddressinfo static

curl_ntlm_core: make hmac_md5 static

http2: make two functions static

http: make http_setup_conn static

connect: make tcpnodelay static

tests: make UNITTEST a thing to mark functions with, so they can be static for
normal builds and non-static for unit test builds

... and mark Curl_shuffle_addr accordingly.

url: make up_free static

setopt: make vsetopt static

curl_endian: make write32_le static

rtsp: make rtsp_connisdead static

warnless: remove unused functions

memdebug: remove one unused function, made another static
2019-02-10 18:38:57 +01:00
Daniel Stenberg
ba243235ec
urldata: rename easy_conn to just conn
We use "conn" everywhere to be a pointer to the connection.

Introduces two functions that "attaches" and "detaches" the connection
to and from the transfer.

Going forward, we should favour using "data->conn" (since a transfer
always only has a single connection or none at all) to "conn->data"
(since a connection can have none, one or many transfers associated with
it and updating conn->data to be correct is error prone and a frequent
reason for internal issues).

Closes #3442
2019-01-11 15:35:13 +01:00
Daniel Stenberg
85f9aa08d9
conncache_unlock: avoid indirection by changing input argument type 2018-12-22 18:14:56 +01:00
Daniel Stenberg
fb445a1e18
disconnect: separate connections and easy handles better
Do not assume/store assocation between a given easy handle and the
connection if it can be avoided.

Long-term, the 'conn->data' pointer should probably be removed as it is a
little too error-prone. Still used very widely though.

Reported-by: masbug on github
Fixes #3391
Closes #3400
2018-12-22 18:14:50 +01:00
Daniel Stenberg
dcd6f81025
snprintf: renamed and we now only use msnprintf()
The function does not return the same value as snprintf() normally does,
so readers may be mislead into thinking the code works differently than
it actually does. A different function name makes this easier to detect.

Reported-by: Tomas Hoger
Assisted-by: Daniel Gustafsson
Fixes #3296
Closes #3297
2018-11-23 08:26:51 +01:00
Daniel Stenberg
81377cace6
vtls: set conn->data when closing TLS
Follow-up to 1b76c38904. The VTLS backends that close down the TLS
layer for a connection still needs a Curl_easy handle for the session_id
cache etc.

Fixes #2764
Closes #2771
2018-07-22 21:17:27 +02:00
Daniel Stenberg
1b76c38904
conn: remove the boolean 'inuse' field
... as the usage needs to be counted.
2018-07-11 23:41:24 +02:00
Daniel Stenberg
f762fec323
url: check Curl_conncache_add_conn return code
... it was previously unchecked in two places and thus errors could
remain undetected and cause trouble.

Closes #2681
2018-06-25 16:36:10 +02:00
Daniel Stenberg
2c15693a3c
url: fix dangling conn->data pointer
By masking sure to use the *current* easy handle with extracted
connections from the cache, and make sure to NULLify the ->data pointer
when the connection is put into the cache to make this mistake easier to
detect in the future.

Reported-by: Will Dietz
Fixes #2669
Closes #2672
2018-06-21 09:44:04 +02:00
Rikard Falkeborn
8ea5d41fe0
strictness: correct {infof, failf} format specifiers
Closes #2623
2018-05-31 11:34:32 +02:00
Daniel Stenberg
41982b6ac9
conncache: fix a return code [regression]
This broke in 07cb27c98e. Make sure to return 'result' properly. Pointed
out by scan-build!
2017-12-12 23:54:35 +01:00
Daniel Stenberg
07cb27c98e
conncache: fix several lock issues
If the lock is released before the dealings with the bundle is over, it may
have changed by another thread in the mean time.

Fixes #2132
Fixes #2151
Closes #2139
2017-12-05 23:21:02 +01:00
Daniel Stenberg
67c55a26d5
share: add support for sharing the connection cache 2017-11-09 11:07:44 +01:00
Daniel Stenberg
cbae73e1dd llist: no longer uses malloc
The 'list element' struct now has to be within the data that is being
added to the list. Removes 16.6% (tiny) mallocs from a simple HTTP
transfer. (96 => 80)

Also removed return codes since the llist functions can't fail now.

Test 1300 updated accordingly.

Closes #1435
2017-04-22 11:25:27 +02:00
Daniel Stenberg
e60fe20fdf llist: replace Curl_llist_alloc with Curl_llist_init
No longer allocate the curl_llist head struct for lists separately.

Removes 17 (15%) tiny allocations in a normal "curl localhost" invoke.

closes #1381
2017-04-04 15:27:45 +02:00
Daniel Stenberg
2066072ff2 conncache: make hashkey avoid malloc
... to make it much faster. Idea developed with primepie on IRC.

Closes #1365
2017-03-29 00:15:46 +02:00
Michael Kaufmann
ddedc1b3f3 string formatting: fix 4 printf-style format strings 2017-02-19 14:13:42 +01:00
Alex Rousskov
cb4e2be7c6 proxy: Support HTTPS proxy and SOCKS+HTTP(s)
* HTTPS proxies:

An HTTPS proxy receives all transactions over an SSL/TLS connection.
Once a secure connection with the proxy is established, the user agent
uses the proxy as usual, including sending CONNECT requests to instruct
the proxy to establish a [usually secure] TCP tunnel with an origin
server. HTTPS proxies protect nearly all aspects of user-proxy
communications as opposed to HTTP proxies that receive all requests
(including CONNECT requests) in vulnerable clear text.

With HTTPS proxies, it is possible to have two concurrent _nested_
SSL/TLS sessions: the "outer" one between the user agent and the proxy
and the "inner" one between the user agent and the origin server
(through the proxy). This change adds supports for such nested sessions
as well.

A secure connection with a proxy requires its own set of the usual SSL
options (their actual descriptions differ and need polishing, see TODO):

  --proxy-cacert FILE        CA certificate to verify peer against
  --proxy-capath DIR         CA directory to verify peer against
  --proxy-cert CERT[:PASSWD] Client certificate file and password
  --proxy-cert-type TYPE     Certificate file type (DER/PEM/ENG)
  --proxy-ciphers LIST       SSL ciphers to use
  --proxy-crlfile FILE       Get a CRL list in PEM format from the file
  --proxy-insecure           Allow connections to proxies with bad certs
  --proxy-key KEY            Private key file name
  --proxy-key-type TYPE      Private key file type (DER/PEM/ENG)
  --proxy-pass PASS          Pass phrase for the private key
  --proxy-ssl-allow-beast    Allow security flaw to improve interop
  --proxy-sslv2              Use SSLv2
  --proxy-sslv3              Use SSLv3
  --proxy-tlsv1              Use TLSv1
  --proxy-tlsuser USER       TLS username
  --proxy-tlspassword STRING TLS password
  --proxy-tlsauthtype STRING TLS authentication type (default SRP)

All --proxy-foo options are independent from their --foo counterparts,
except --proxy-crlfile which defaults to --crlfile and --proxy-capath
which defaults to --capath.

Curl now also supports %{proxy_ssl_verify_result} --write-out variable,
similar to the existing %{ssl_verify_result} variable.

Supported backends: OpenSSL, GnuTLS, and NSS.

* A SOCKS proxy + HTTP/HTTPS proxy combination:

If both --socks* and --proxy options are given, Curl first connects to
the SOCKS proxy and then connects (through SOCKS) to the HTTP or HTTPS
proxy.

TODO: Update documentation for the new APIs and --proxy-* options.
Look for "Added in 7.XXX" marks.
2016-11-24 23:41:44 +01:00
Daniel Stenberg
811a693b80 strcasecompare: all case insensitive string compares ignore locale now
We had some confusions on when each function was used. We should not act
differently on different locales anyway.
2016-10-31 08:46:35 +01:00
Daniel Stenberg
434f8d0389 internals: rename the SessionHandle struct to Curl_easy 2016-06-22 10:28:41 +02:00
Daniel Stenberg
5823179523 connections: non-HTTP proxies on different ports aren't reused either
Reported-by: Oleg Pudeyev and fuchaoqun

Fixes #648
2016-05-03 08:18:15 +02:00
Daniel Stenberg
4f45240bc8 lib: include curl_printf.h as one of the last headers
curl_printf.h defines printf to curl_mprintf, etc. This can cause
problems with external headers which may use
__attribute__((format(printf, ...))) markers etc.

To avoid that they cause problems with system includes, we include
curl_printf.h after any system headers. That makes the three last
headers to always be, and we keep them in this order:

 curl_printf.h
 curl_memory.h
 memdebug.h

None of them include system headers, they all do funny #defines.

Reported-by: David Benjamin

Fixes #743
2016-04-29 22:32:49 +02:00
Michael Kaufmann
cd8d236245 news: CURLOPT_CONNECT_TO and --connect-to
Makes curl connect to the given host+port instead of the host+port found
in the URL.
2016-04-17 23:50:59 +02:00