Commit Graph

672 Commits

Author SHA1 Message Date
Daniel Stenberg
31a41d45b7
multi: have curl_multi_remove_handle close CONNECT_ONLY transfer
Ẃhen it has been used in the multi interface, it is otherwise left in
the connection cache, can't be reused and nothing will close them since
the easy handle loses the association with the multi handle and thus the
connection cache - until the multi handle is closed or it gets pruned
because the cache is full.

Reported-by: Dominik Thalhammer
Fixes #9335
Closes #9342
2022-08-23 13:44:11 +02:00
Daniel Stenberg
4c57fdcf87
multi: fix the return code from Curl_pgrsDone()
It does not return a CURLcode. Detected by the icc compiler warning
"enumerated type mixed with another type"

Closes #9179
2022-07-23 13:38:47 +02:00
max.mehl
ad9bc5976d
copyright: make repository REUSE compliant
Add licensing and copyright information for all files in this repository. This
either happens in the file itself as a comment header or in the file
`.reuse/dep5`.

This commit also adds a Github workflow to check pull requests and adapts
copyright.pl to the changes.

Closes #8869
2022-06-13 09:13:00 +02:00
Daniel Stenberg
5912da253b
select: return error from "lethal" poll/select errors
Adds two new error codes: CURLE_UNRECOVERABLE_POLL and
CURLM_UNRECOVERABLE_POLL one each for the easy and the multi interfaces.

Reported-by: Harry Sintonen
Fixes #8921
Closes #8961
2022-06-08 11:07:01 +02:00
Adam Rosenfield
e07a9b668a
conn: fix typo 'connnection' -> 'connection' in two function names
Closes #8759
2022-04-27 11:50:13 +02:00
Daniel Stenberg
6968fb9d54
lib: remove exclamation marks
... from infof() and failf() calls. Make them less attention seeking.

Closes #8713
2022-04-16 11:55:05 +02:00
Daniel Stenberg
d1e4a67734
header api: add curl_easy_header and curl_easy_nextheader
Add test 1940 to 1946 to verify.

Closes #8593
2022-03-22 08:24:22 +01:00
Jay Satiro
e0dc9765a7 multi: allow user callbacks to call curl_multi_assign
Several years ago a change was made to block user callbacks from calling
back into the API when not supported (recursive calls). One of the calls
blocked was curl_multi_assign. Recently the blocking was extended to the
multi interface API, however curl_multi_assign may need to be called
from within those user callbacks (eg CURLMOPT_SOCKETFUNCTION).

I can't think of any callback where it would be unsafe to call
curl_multi_assign so I removed the restriction entirely.

Reported-by: Michael Wallner

Ref: https://github.com/curl/curl/commit/b46cfbc
Ref: https://github.com/curl/curl/commit/340bb19

Fixes https://github.com/curl/curl/issues/8480
Closes https://github.com/curl/curl/pull/8483
2022-02-21 03:23:12 -05:00
Sebastian Sterk
c6218f3723
multi: grammar fix in comment
After 'must', the verb is used without 'to'. Correct: "must" or "have
to"

Closes #8368
2022-02-01 09:37:26 +01:00
luminixinc on github
3c798b1db3
multi: remember connection_id before returning connection to pool
Fix a bug that does not require a new CVE as discussed on hackerone.com.
Previously `connection_id` was accessed after returning connection to
the shared pool.

Bug: https://hackerone.com/reports/1463013
Closes #8355
2022-01-31 08:55:22 +01:00
Daniel Stenberg
e74a6b7b72
url: make Curl_disconnect return void
1. The function would only ever return CURLE_OK anyway
 2. Only one caller actually used the return code
 3. Most callers did (void)Curl_disconnect()

Closes #8303
2022-01-20 09:07:30 +01:00
Daniel Stenberg
340bb1905f
multi: set in_callback for multi interface callbacks
This makes most libcurl functions return error if called from within a
callback using the same multi handle. For example timer or socket
callbacks calling curl_multi_socket_action.

Reported-by: updatede on github
Fixes #8282
Closes #8286
2022-01-17 16:44:45 +01:00
Daniel Stenberg
254f7bd78a
hash: lazy-alloc the table in Curl_hash_add()
This makes Curl_hash_init() infallible which saves error paths.

Closes #8132
2021-12-10 23:16:43 +01:00
Daniel Stenberg
e43ad4b474
multi: cleanup the socket hash when destroying it
Since each socket hash entry may themselves have a hash table in them,
the destroying of the socket hash needs to make sure all the subhashes
are also correctly destroyed to avoid leaking memory.

Fixes #8129
Closes #8131
2021-12-10 17:02:04 +01:00
Daniel Stenberg
2b3dd01b77
multi: handle errors returned from socket/timer callbacks
The callbacks were partially documented to support this. Now the
behavior is documented and returning error from either of these
callbacks will effectively kill all currently ongoing transfers.

Added test 530 to verify

Reported-by: Marcelo Juchem
Fixes #8083
Closes #8089
2021-12-06 14:55:52 +01:00
Daniel Stenberg
f0b7099a10
multi: shut down CONNECT in Curl_detach_connnection
... to prevent a lingering pointer that would lead to a double-free.

Added test 1939 to verify.

Reported-by: Stephen M. Coakley
Fixes #7982
Closes #7986
2021-11-11 08:42:31 +01:00
Eddie Lumpkin
8e701cc978
lib: fixing comment spelling typos in lib files
Closes #7894
Signed-off-by: ewlumpkin <ewlumpkin@gmail.com>
2021-10-22 22:53:44 +02:00
Max Dymond
a517378de5
CURLOPT_PREREQFUNCTION: add new callback
Triggered before a request is made but after a connection is set up

Changes:

- callback: Update docs and callback for pre-request callback
- Add documentation for CURLOPT_PREREQDATA and CURLOPT_PREREQFUNCTION,
- Add redirect test and callback failure test
- Note that the function may be called multiple times on a redirection
- Disable new 2086 test due to Windows weirdness

Closes #7477
2021-09-27 17:16:43 +02:00
Rikard Falkeborn
e75be2c4b2
cleanup: constify unmodified static structs
Constify a number of static structs that are never modified. Make them
const to show this.

Closes #7759
2021-09-23 12:54:35 +02:00
Daniel Stenberg
d5a70e77b2
curl_multi_fdset: make FD_SET() not operate on sockets out of range
The VALID_SOCK() macro was made to only check for FD_SETSIZE if curl was
built to use select(), even though the curl_multi_fdset() function
always and unconditionally uses FD_SET and needs the check.

Reported-by: 0xee on github
Fixes #7718
Closes #7719
2021-09-15 14:34:00 +02:00
Marcel Raad
1b70748e86
multi: fix compiler warning with CURL_DISABLE_WAKEUP
`use_wakeup` is unused in this case.

Closes https://github.com/curl/curl/pull/7661
2021-09-02 15:52:00 +02:00
User Sg
6a66f72829 multi: fix crash in curl_multi_wait / curl_multi_poll
Appears to have been caused by 51c0ebc (precedes 7.77.0) which added a
VALID_SOCK check to one of the loops through the sockets but not the
other.

Reported-by: sylgal@users.noreply.github.com
Authored-by: sylgal@users.noreply.github.com

Fixes https://github.com/curl/curl/issues/7379
Closes https://github.com/curl/curl/pull/7389
2021-07-16 14:17:47 -04:00
Daniel Stenberg
e7416cfd2b
infof: remove newline from format strings, always append it
- the data needs to be "line-based" anyway since it's also passed to the
  debug callback/application

- it makes infof() work like failf() and consistency is good

- there's an assert that triggers on newlines in the format string

- Also removes a few instances of "..."

- Removes the code that would append "..." to the end of the data *iff*
  it was truncated in infof()

Closes #7357
2021-07-07 22:54:01 +02:00
Richard Whitehouse
0842175fa4
multi: alter transfer timeout ordering
- Check whether a connection has succeded before checking whether it's
  timed out.

  This means if we've connected quickly, but subsequently been
  descheduled, we allow the connection to succeed. Note, if we timeout,
  but between checking the timeout, and connecting to the server the
  connection succeeds, we will allow it to go ahead. This is viewed as
  an acceptable trade off.

- Add additional failf logging around failed connection attempts to
  propogate the cause up to the caller.

Co-Authored-by: Martin Howarth
Closes #7178
2021-06-24 15:51:39 +02:00
Daniel Stenberg
9accc48850
multi: do not switch off connect_only flag when closing
... as it made protocol specific disconnect commands wrongly get used.

Bug: https://curl.se/mail/lib-2021-06/0024.html
Reported-by: Aleksander Mazur
Closes #7288
2021-06-24 09:07:34 +02:00
Daniel Stenberg
77bc35901f
multi: add scan-build-6 work-around in curl_multi_fdset
scan-build-6 otherwise warns, saying: warning: The left operand of '>='
is a garbage value otherwise, which is false.

Later scan-builds don't claim this on the same code.

Closes #7248
2021-06-13 23:43:01 +02:00
Harry Sintonen
7f4a9a9b2a
openssl: associate/detach the transfer from connection
CVE-2021-22901

Bug: https://curl.se/docs/CVE-2021-22901.html
2021-05-24 13:15:10 +02:00
Daniel Stenberg
0c55fbab45
conn: add 'attach' to protocol handler, make libssh2 use it
The libssh2 backend has SSH session associated with the connection but
the callback context is the easy handle, so when a connection gets
attached to a transfer, the protocol handler now allows for a custom
function to get used to set things up correctly.

Reported-by: Michael O'Farrell
Fixes #6898
Closes #7078
2021-05-17 17:57:50 +02:00
Daniel Stenberg
51c0ebcff2
http: deal with partial CONNECT sends
Also added 'CURL_SMALLSENDS' to make Curl_write() send short packets,
which helped verifying this even more.

Add test 363 to verify.

Reported-by: ustcqidi on github
Fixes #6950
Closes #7024
2021-05-08 10:49:16 +02:00
Daniel Stenberg
063d3f3b96
tidy-up: make conditional checks more consistent
... remove '== NULL' and '!= 0'

Closes #6912
2021-04-22 09:10:17 +02:00
Marc Hoersken
b36442b243
multi: fix slow write/upload performance on Windows
Reset FD_WRITE by sending zero bytes which is permissible
and will be treated by implementations as successful send.

Without this we won't be notified in case a socket is still
writable if we already received such a notification and did
not send any data afterwards on the socket. This would lead
to waiting forever on a writable socket being writable again.

Assisted-by: Tommy Odom
Reviewed-by: Jay Satiro
Reviewed-by: Marcel Raad
Tested-by: tmkk on github

Bug: #6146
Closes #6245
2021-04-21 20:32:24 +02:00
Marc Hoersken
e92998a312
multi: reduce Win32 API calls to improve performance
1. Consolidate pre-checks into a single Curl_poll call:

This is an attempt to restructure the code in Curl_multi_wait
in such a way that less syscalls are made by removing individual
calls to Curl_socket_check via SOCKET_READABLE/SOCKET_WRITABLE.

2. Avoid resetting the WinSock event multiple times:

We finally call WSAResetEvent anyway, so specifying it as
an optional parameter to WSAEnumNetworkEvents is redundant.

3. Wakeup directly in case no sockets are being monitoring:

Fix the WinSock based implementation to skip extra waiting by
not sleeping in case no sockets are to be waited on and just
the WinSock event is being monitored for wakeup functionality.

Assisted-by: Tommy Odom
Reviewed-by: Jay Satiro
Reviewed-by: Marcel Raad

Bug: #6146
Closes #6245
2021-04-21 20:32:14 +02:00
Marc Hoersken
bcca174cfa
Revert "Revert 'multi: implement wait using winsock events'"
This reverts commit 2260e0ebe6,
also restoring previous follow up changes which were reverted.

Authored-by: rcombs on github
Authored-by: Marc Hörsken
Reviewed-by: Jay Satiro
Reviewed-by: Marcel Raad

Restores #5634
Reverts #6281
Part of #6245
2021-04-21 20:29:36 +02:00
Daniel Stenberg
9c18c0b4dd
multi: don't close connection HTTP_1_1_REQUIRED
The ConnectionExists() function will note that the new transfer wants
less then h2 and that it can't multiplex it and therefor opt to open a
new connection instead.
2021-04-21 08:20:24 +02:00
Daniel Stenberg
605e842355
http2: move the stream error field to the per-transfer storage
Storing a stream error in the per-connection struct was an error that lead to
race conditions as subsequent stream handling could overwrite the error code
before it was used for the stream with the actual problem.

Closes #6910
2021-04-21 08:20:24 +02:00
Daniel Stenberg
b5726e5549
send_speed: simplify the checks for if a speed limit is set
... as we know the value cannot be set to negative: enforced by
setopt()
2021-03-27 12:38:28 +01:00
Daniel Stenberg
95cbcec8f9
urldata: merge "struct DynamicStatic" into "struct UrlState"
Both were used for the same purposes and there was no logical separation
between them. Combined, this also saves 16 bytes in less holes in my
test build.

Closes #6798
2021-03-26 23:19:20 +01:00
Daniel Stenberg
7b6bfd2d78
multi: close the connection when h2=>h1 downgrading
Otherwise libcurl is likely to reuse the connection again in the next
attempt since the connection reuse logic doesn't take downgrades into
account.

Reported-by: Anthony Ramine
Fixes #6788
Closes #6793
2021-03-25 16:18:13 +01:00
ejanchivdorj
a2bbc3ac8c
multi: update pending list when removing handle
when removing a handle, most of the lists are updated but pending list
is not updated. Updating now.

Closes #6713
2021-03-10 14:05:02 +01:00
Daniel Stenberg
0c7d111f4e
urldata: make 'actions[]' use unsigned char instead of int
... as it only needs a few bits per index anyway.

Reviewed-by: Daniel Gustafsson
Closes #6648
2021-02-24 14:16:11 +01:00
Daniel Stenberg
bf601472a0
multi: do once-per-transfer inits in before_perform in DID state
... since the state machine might go to RATELIMITING and then back to
PERFORMING doing once-per-transfer inits in that function is wrong and
it caused problems with receiving chunked HTTP and it set the
PRETRANSFER time much too often...

Regression from b68dc34af3 (shipped in 7.75.0)

Reported-by: Amaury Denoyelle
Fixes #6640
Closes #6641
2021-02-22 15:54:46 +01:00
Daniel Stenberg
94719e7285
multi: rename the multi transfer states
While working on documenting the states it dawned on me that step one is
to use more descriptive names on the states. This also changes prefix on
the states to make them shorter in the source.

State names NOT ending with *ing are transitional ones.

Closes #6612
2021-02-16 16:42:46 +01:00
Daniel Stenberg
cfff12a0b3
lib: remove 'conn->data' completely
The Curl_easy pointer struct entry in connectdata is now gone. Just
before commit 215db086e0 landed on January 8, 2021 there were 919
references to conn->data.

Closes #6608
2021-02-16 10:48:57 +01:00
Daniel Stenberg
88dd1a8a11
urldata: don't touch data->set.httpversion at run-time
Rename it to 'httpwant' and make a cloned field in the state struct as
well for run-time updates.

Also: refuse non-supported HTTP versions. Verified with test 129.

Closes #6585
2021-02-12 08:13:37 +01:00
Jacob Hoffman-Andrews
246399a874
vtls: initial implementation of rustls backend
This adds a new TLS backend, rustls. It uses the C-to-rustls bindings
from https://github.com/abetterinternet/crustls.

Rustls is at https://github.com/ctz/rustls/.

There is still a fair bit to be done, like sending CloseNotify on
connection shutdown, respecting CAPATH, and properly indicating features
like "supports TLS 1.3 ciphersuites." But it works well enough to make
requests and receive responses.

Blog post for context:
https://www.abetterinternet.org/post/memory-safe-curl/

Closes #6350
2021-02-09 11:06:18 +01:00
Daniel Stenberg
1dc8aa870e
hostip/proxy: remove conn->data use
Closes #6513
2021-01-26 10:04:47 +01:00
Daniel Stenberg
8335c6417e
hostip: remove conn->data from resolver functions
This also moves the 'async' struct from the connectdata struct into the
Curl_easy struct, which seems like a better home for it.

Closes #6497
2021-01-22 08:25:09 +01:00
Erik Olsson
0a5827571f lib: save a bit of space with some structure packing
- Reorder some internal struct members so that less padding is used.

This is an attempt at saving a bit of space by packing some structs
(using pahole to find the holes) where it might make sense to do
so without losing readability.

I.e., I tried to avoid separating fields that seem grouped
together (like the cwd... fields in struct ftp_conn for instance).
Also abstained from touching fields behind conditional macros as
that quickly can get complicated.

Closes https://github.com/curl/curl/pull/6483
2021-01-20 22:52:34 -05:00
Daniel Stenberg
942cf12c2f
urldata: make magic be the first struct field
By making the `magic` identifier the same size and at the same place
within the structs (easy, multi, share), libcurl will be able to more
reliably detect and safely error out if an application passes in the
wrong handle to APIs. Easier to detect and less likely to cause crashes
if done.

Such mixups can't be detected at compile-time due to them being
typedefed void pointers - unless `CURL_STRICTER` is defined.

Closes #6484
2021-01-20 08:51:01 +01:00
Daniel Stenberg
219d9f8620
transfer: remove conn->data use
Closes #6486
2021-01-19 17:07:02 +01:00