Commit Graph

96 Commits

Author SHA1 Message Date
Daniel Stenberg
e15e51384a
http: move header storage to Curl_easy from connectdata
Since the connection can be used by many independent requests (using
HTTP/2 or HTTP/3), things like user-agent and other transfer-specific
data MUST NOT be kept connection oriented as it could lead to requests
getting the wrong string for their requests. This struct data was
lingering like this due to old HTTP1 legacy thinking where it didn't
mattered..

Fixes #5566
Closes #5567
2020-06-15 22:56:25 +02:00
Daniel Stenberg
dae126ff12
http_proxy: ported to use dynbuf instead of a static size buffer
Removes a 16K static buffer from the easy handle. Simplifies the code.
2020-05-04 10:41:06 +02:00
Daniel Stenberg
ed35d6590e
dynbuf: introduce internal generic dynamic buffer functions
A common set of functions instead of many separate implementations for
creating buffers that can grow when appending data to them. Existing
functionality has been ported over.

In my early basic testing, the total number of allocations seem at
roughly the same amount as before, possibly a few less.

See docs/DYNBUF.md for a description of the API.

Closes #5300
2020-05-04 10:40:39 +02:00
Daniel Stenberg
c7bc689fc3
conn: always set bits.close with connclose()
Closes #4690
2019-12-09 17:17:42 +01:00
Daniel Stenberg
f0f053fed0
chunked-encoding: stop hiding the CURLE_BAD_CONTENT_ENCODING error
Unknown content-encoding would get returned as CURLE_WRITE_ERROR if the
response is chunked-encoded.

Reported-by: Ilya Kosarev
Fixes #4310
Closes #4449
2019-10-02 07:46:05 +02:00
Daniel Stenberg
cc95dbd64f
http_proxy: part of conditional expression is always true: !error
Fixes warning detected by PVS-Studio
Fixes #4374
2019-09-20 08:07:28 +02:00
Sergey Ogryzkov
f9b60fb6f8
NTLM: reset proxy "multipass" state when CONNECT request is done
Closes #3972
2019-06-02 23:11:33 +02:00
Daniel Stenberg
ba243235ec
urldata: rename easy_conn to just conn
We use "conn" everywhere to be a pointer to the connection.

Introduces two functions that "attaches" and "detaches" the connection
to and from the transfer.

Going forward, we should favour using "data->conn" (since a transfer
always only has a single connection or none at all) to "conn->data"
(since a connection can have none, one or many transfers associated with
it and updating conn->data to be correct is error prone and a frequent
reason for internal issues).

Closes #3442
2019-01-11 15:35:13 +01:00
Daniel Stenberg
55dbcb061d
http: made Curl_add_buffer functions take a pointer-pointer
... so that they can clear the original pointer on failure, which makes
the error-paths and their cleanups easier.

Closes #2992
2018-09-16 23:22:37 +02:00
Rikard Falkeborn
64c01db0c9
http_proxy: Remove unused macro SELECT_TIMEOUT
Usage was removed in 5113ad0424.

Closes #2852
2018-08-09 12:37:16 +02:00
Daniel Stenberg
5ae22272d1
Curl_debug: remove dead printhost code
The struct field is never set (since 5e0d9aea3) so remove the use of it
and remove the connectdata pointer from the prototype.

Reported-by: Tejas
Bug: https://curl.haxx.se/mail/lib-2018-06/0054.html
Closes #2647
2018-06-12 16:00:27 +02:00
Marian Klymov
c45360d463
cppcheck: fix warnings
- Get rid of variable that was generating false positive warning
(unitialized)

- Fix issues in tests

- Reduce scope of several variables all over

etc

Closes #2631
2018-06-11 11:14:48 +02:00
Rikard Falkeborn
8ea5d41fe0
strictness: correct {infof, failf} format specifiers
Closes #2623
2018-05-31 11:34:32 +02:00
Rikard Falkeborn
eb49683e55
lib: Fix format specifiers 2018-05-14 09:42:27 +02:00
Daniel Stenberg
8123560d44
HTTP: allow "header;" to replace an internal header with a blank one
Reported-by: Michael Kaufmann
Fixes #2357
Closes #2362
2018-03-11 11:46:10 +01:00
Zachary Seguin
55e609890f
CONNECT: keep close connection flag in http_connect_state struct
Fixes #2088
Closes #2157
2017-12-07 09:48:33 +01:00
Daniel Stenberg
7ee59512f8
timeleft: made two more users of Curl_timeleft use timediff_t 2017-10-29 13:13:23 +01:00
Daniel Stenberg
6b84438d9a
code style: use spaces around equals signs 2017-09-11 09:29:50 +02:00
Daniel Stenberg
9ef50ee0a4
http-proxy: when not doing CONNECT, that phase is done immediately
`conn->connect_state` is NULL when doing a regular non-CONNECT request
over the proxy and should therefor be considered complete at once.

Fixes #1853
Closes #1862
Reported-by: Lawrence Wagerfield
2017-09-07 16:11:38 +02:00
Daniel Stenberg
3130414ce7
http-proxy: treat all 2xx as CONNECT success
Added test 1904 to verify.

Reported-by: Lawrence Wagerfield
Fixes #1859
Closes #1860
2017-09-05 09:47:46 +02:00
Maksim Stsepanenka
801d8765ca http_proxy: fix build error for CURL_DOES_CONVERSIONS
Closes https://github.com/curl/curl/pull/1793
2017-08-16 14:41:31 -04:00
Daniel Stenberg
ff50fe0348
strtoofft: reduce integer overflow risks globally
... make sure we bail out on overflows.

Reported-by: Brian Carpenter
Closes #1758
2017-08-14 23:33:41 +02:00
Marcel Raad
edafd52be1
http_proxy: fix build with http and proxy
After deff7de0eb, the build without
CURL_DISABLE_PROXY and CURL_DISABLE_HTTP was failing because of missing
includes.
2017-06-18 15:18:15 +02:00
Marcel Raad
deff7de0eb
http_proxy: fix compiler warning
With CURL_DISABLE_PROXY or CURL_DISABLE_HTTP, GCC complained about a
missing prototype for Curl_connect_free.
2017-06-18 12:53:54 +02:00
Daniel Stenberg
01811b6740 http-proxy: fix chunked-encoded CONNECT responses
Regression since 5113ad0424.

... and remove 'flaky' from test 1061 again

Closes #1579
2017-06-16 21:56:56 +02:00
Daniel Stenberg
12ccaed2a5 http-proxy: deal with EAGAIN
... the previous code would reset the header length wrongly (since
5113ad0424). This makes test 1060 reliable again.

Also: make sws send even smaller chunks of data to increase the
likeliness of this happening.
2017-06-16 17:22:32 +02:00
Daniel Stenberg
0d39e5cc95 http-proxy: fix build with --disable-proxy or --disable-http
Reported-by: Dan Fandrich
2017-06-16 00:02:57 +02:00
Daniel Stenberg
5113ad0424 http-proxy: do the HTTP CONNECT process entirely non-blocking
Mentioned as a problem since 2007 (8f87c15bda) and of course it
existed even before that.

Closes #1547
2017-06-14 23:43:52 +02:00
Daniel Stenberg
0cab3a394a http-proxy: use a dedicated CONNECT response buffer
To make it suitably independent of the receive buffer and its flexible
size.
2017-05-01 22:55:29 +02:00
Daniel Stenberg
55c3c02e59 http-proxy: remove unused argument from Curl_proxyCONNECT() 2017-04-26 00:34:22 +02:00
Sylvestre Ledru
66de563482 Improve code readbility
... by removing the else branch after a return, break or continue.

Closes #1310
2017-03-13 23:11:45 +01:00
Desmond O. Chang
d2bcf1e3e2 url: add option CURLOPT_SUPPRESS_CONNECT_HEADERS
- Add new option CURLOPT_SUPPRESS_CONNECT_HEADERS to allow suppressing
  proxy CONNECT response headers from the user callback functions
  CURLOPT_HEADERFUNCTION and CURLOPT_WRITEFUNCTION.

- Add new tool option --suppress-connect-headers to expose
  CURLOPT_SUPPRESS_CONNECT_HEADERS and allow suppressing proxy CONNECT
  response headers from --dump-header and --include.

Assisted-by: Jay Satiro
Assisted-by: CarloCannas@users.noreply.github.com
Closes https://github.com/curl/curl/pull/783
2017-03-12 01:32:33 -05:00
Jay Satiro
ec1d0ed1c1 http_proxy: Ignore TE and CL in CONNECT 2xx responses
A client MUST ignore any Content-Length or Transfer-Encoding header
fields received in a successful response to CONNECT.
"Successful" described as: 2xx (Successful). RFC 7231 4.3.6

Prior to this change such a case would cause an error.

In some ways this bug appears to be a regression since c50b878. Prior to
that libcurl may have appeared to function correctly in such cases by
acting on those headers instead of causing an error. But that behavior
was also incorrect.

Bug: https://github.com/curl/curl/issues/1317
Reported-by: mkzero@users.noreply.github.com
2017-03-11 18:22:42 -05:00
Michael Kaufmann
2f8d0df085 proxy: fix hostname resolution and IDN conversion
Properly resolve, convert and log the proxy host names.
Support the "--connect-to" feature for SOCKS proxies and for passive FTP
data transfers.

Follow-up to cb4e2be

Reported-by: Jay Satiro
Fixes https://github.com/curl/curl/issues/1248
2017-02-18 15:04:43 +01:00
Daniel Stenberg
e2e182227a http_proxy: avoid freeing static memory
Follow up to 7fe81ec298: make sure 'host' is either NULL or malloced.
2017-02-09 10:13:28 +01:00
Cameron MacMinn
7fe81ec298 http_proxy: Fix tiny memory leak upon edge case connecting to proxy
Fixes #1255
2017-02-09 09:51:42 +01:00
Jay Satiro
d00f2a8f2e http_proxy: Fix proxy CONNECT hang on pending data
- Check for pending data before waiting on the socket.

Bug: https://github.com/curl/curl/issues/1156
Reported-by: Adam Langley
2016-12-19 02:26:52 -05:00
Daniel Stenberg
b228d2952b checksrc: stricter no-space-before-paren enforcement
In order to make the code style more uniform everywhere
2016-12-13 23:39:11 +01:00
Daniel Stenberg
74595b223d http_proxy: simplify CONNECT response reading
Since it now reads responses one byte a time, a loop could be removed
and it is no longer limited to get the whole response within 16K, it is
now instead only limited to 16K maximum header line lengths.
2016-12-01 16:18:52 +01:00
Daniel Stenberg
3ea3518429 CONNECT: read responses one byte at a time
... so that it doesn't read data that is actually coming from the
remote. 2xx responses have no body from the proxy, that data is from the
peer.

Fixes #1132
2016-12-01 16:18:36 +01:00
Daniel Stenberg
c50b878c15 CONNECT: reject TE or CL in 2xx responses
A server MUST NOT send any Transfer-Encoding or Content-Length header
fields in a 2xx (Successful) response to CONNECT. (RFC 7231 section
4.3.6)

Also fixes the three test cases that did this.
2016-12-01 16:18:36 +01:00
Okhin Vasilij
c6da05a5ec HTTPS-proxy: fixed mbedtls and polishing 2016-11-24 23:41:45 +01:00
Alex Rousskov
cb4e2be7c6 proxy: Support HTTPS proxy and SOCKS+HTTP(s)
* HTTPS proxies:

An HTTPS proxy receives all transactions over an SSL/TLS connection.
Once a secure connection with the proxy is established, the user agent
uses the proxy as usual, including sending CONNECT requests to instruct
the proxy to establish a [usually secure] TCP tunnel with an origin
server. HTTPS proxies protect nearly all aspects of user-proxy
communications as opposed to HTTP proxies that receive all requests
(including CONNECT requests) in vulnerable clear text.

With HTTPS proxies, it is possible to have two concurrent _nested_
SSL/TLS sessions: the "outer" one between the user agent and the proxy
and the "inner" one between the user agent and the origin server
(through the proxy). This change adds supports for such nested sessions
as well.

A secure connection with a proxy requires its own set of the usual SSL
options (their actual descriptions differ and need polishing, see TODO):

  --proxy-cacert FILE        CA certificate to verify peer against
  --proxy-capath DIR         CA directory to verify peer against
  --proxy-cert CERT[:PASSWD] Client certificate file and password
  --proxy-cert-type TYPE     Certificate file type (DER/PEM/ENG)
  --proxy-ciphers LIST       SSL ciphers to use
  --proxy-crlfile FILE       Get a CRL list in PEM format from the file
  --proxy-insecure           Allow connections to proxies with bad certs
  --proxy-key KEY            Private key file name
  --proxy-key-type TYPE      Private key file type (DER/PEM/ENG)
  --proxy-pass PASS          Pass phrase for the private key
  --proxy-ssl-allow-beast    Allow security flaw to improve interop
  --proxy-sslv2              Use SSLv2
  --proxy-sslv3              Use SSLv3
  --proxy-tlsv1              Use TLSv1
  --proxy-tlsuser USER       TLS username
  --proxy-tlspassword STRING TLS password
  --proxy-tlsauthtype STRING TLS authentication type (default SRP)

All --proxy-foo options are independent from their --foo counterparts,
except --proxy-crlfile which defaults to --crlfile and --proxy-capath
which defaults to --capath.

Curl now also supports %{proxy_ssl_verify_result} --write-out variable,
similar to the existing %{ssl_verify_result} variable.

Supported backends: OpenSSL, GnuTLS, and NSS.

* A SOCKS proxy + HTTP/HTTPS proxy combination:

If both --socks* and --proxy options are given, Curl first connects to
the SOCKS proxy and then connects (through SOCKS) to the HTTP or HTTPS
proxy.

TODO: Update documentation for the new APIs and --proxy-* options.
Look for "Added in 7.XXX" marks.
2016-11-24 23:41:44 +01:00
Marcel Raad
21aa32d30d lib: fix compiler warnings after de4de4e3c7
Visual C++ now complains about implicitly casting time_t (64-bit) to
long (32-bit). Fix this by changing some variables from long to time_t,
or explicitly casting to long where the public interface would be
affected.

Closes #1131
2016-11-18 10:11:55 +01:00
Daniel Stenberg
811a693b80 strcasecompare: all case insensitive string compares ignore locale now
We had some confusions on when each function was used. We should not act
differently on different locales anyway.
2016-10-31 08:46:35 +01:00
Daniel Stenberg
8a6e89a9eb select: switch to macros in uppercase
Curl_select_ready() was the former API that was replaced with
Curl_select_check() a while back and the former arg setup was provided
with a define (in order to leave existing code unmodified).

Now we instead offer SOCKET_READABLE and SOCKET_WRITABLE for the most
common shortcuts where only one socket is checked. They're also more
visibly macros.
2016-10-18 11:05:45 +02:00
Daniel Stenberg
3533def3d5 http2: make sure stream errors don't needlessly close the connection
With HTTP/2 each transfer is made in an indivial logical stream over the
connection, making most previous errors that caused the connection to get
forced-closed now instead just kill the stream and not the connection.

Fixes #941
2016-08-28 16:44:49 +02:00
Daniel Stenberg
93b0d907d5 Revert "Proxy-Connection: stop sending this header by default"
This reverts commit 113f04e664.
2016-08-16 08:36:04 +02:00
Daniel Stenberg
434f8d0389 internals: rename the SessionHandle struct to Curl_easy 2016-06-22 10:28:41 +02:00
Daniel Stenberg
4f45240bc8 lib: include curl_printf.h as one of the last headers
curl_printf.h defines printf to curl_mprintf, etc. This can cause
problems with external headers which may use
__attribute__((format(printf, ...))) markers etc.

To avoid that they cause problems with system includes, we include
curl_printf.h after any system headers. That makes the three last
headers to always be, and we keep them in this order:

 curl_printf.h
 curl_memory.h
 memdebug.h

None of them include system headers, they all do funny #defines.

Reported-by: David Benjamin

Fixes #743
2016-04-29 22:32:49 +02:00