Commit Graph

367 Commits

Author SHA1 Message Date
Daniel Stenberg
1485e89213
misc: reduce struct and struct field sizes
- by using BIT() instead of bool
- imap: shrink struct
- ftp: make state 'unsigned char'
- ftp: sort ftp_conn struct entries on size
- urldata: use smaller fields for SSL version info storage
- pop3: reduce the pop3_conn struct size
- smtp: reduce the size of the smtp structs

Closes #10186
2023-01-04 15:36:36 +01:00
Daniel Stenberg
2bc1d775f5
copyright: update all copyright lines and remove year ranges
- they are mostly pointless in all major jurisdictions
- many big corporations and projects already don't use them
- saves us from pointless churn
- git keeps history for us
- the year range is kept in COPYING

checksrc is updated to allow non-year using copyright statements

Closes #10205
2023-01-03 09:19:21 +01:00
Stefan Eissing
71b7e01610
lib: connect/h2/h3 refactor
Refactoring of connection setup and happy eyeballing. Move
nghttp2. ngtcp2, quiche and msh3 into connection filters.

 - eyeballing cfilter that uses sub-filters for performing parallel connects
 - socket cfilter for all transport types, including QUIC
 - QUIC implementations in cfilter, can now participate in eyeballing
 - connection setup is more dynamic in order to adapt to what filter did
   really connect.  Relevant to see if a SSL filter needs to be added or
   if SSL has already been provided
 - HTTP/3 test cases similar to HTTP/2
 - multiuse of parallel transfers for HTTP/3, tested for ngtcp2 and quiche

 - Fix for data attach/detach in VTLS filters that could lead to crashes
   during parallel transfers.
 - Eliminating setup() methods in cfilters, no longer needed.
 - Improving Curl_conn_is_alive() to replace Curl_connalive() and
   integrated ssl alive checks into cfilter.
 - Adding CF_CNTRL_CONN_INFO_UPDATE to tell filters to update
   connection into and persist it at the easy handle.

 - Several more cfilter related cleanups and moves:
   - stream_weigth and dependency info is now wrapped in struct
     Curl_data_priority
   - Curl_data_priority members depend is available in HTTP2|HTTP3
   - Curl_data_priority members depend on NGHTTP2 support
   - handling init/reset/cleanup of priority part of url.c
   - data->state.priority same struct, but shallow copy for compares only

 - PROTOPT_STREAM has been removed
   - Curl_conn_is_mulitplex() now available to check on capability

 - Adding query method to connection filters.
   - ngtcp2+quiche: implementing query for max concurrent transfers.

 - Adding is_alive and keep_alive cfilter methods. Adding DATA_SETUP event.
   - setting keepalive timestamp on connect
   - DATA_SETUP is called after the connection has been completely
     setup (but may not connected yet) to allow filters to initialize
     data members they use.

 - there is no socket to be had with msh3, it is unclear how select
   shall work

 - manual test via "curl --http3 https://curl.se" fail with "empty
   reply from server".

 - Various socket/conn related cleanups:
   - Curl_socket is now Curl_socket_open and in cf-socket.c
   - Curl_closesocket is now Curl_socket_close and in cf-socket.c
   - Curl_ssl_use has been replaced with Cur_conn_is_ssl
   - Curl_conn_tcp_accepted_set has been split into
     Curl_conn_tcp_listen_set and Curl_conn_tcp_accepted_set
     with a clearer purpose

Closes #10141
2022-12-30 16:43:19 +01:00
Stefan Eissing
b42156b825
cfilter: improve SSL connection checks
- fixes `Curl_ssl_cf_get_ssl()` to detect also the first filter instance
  as ssl (refs #10053)

- replaces `Curl_ssl_use()` with the correct `Curl_conn_is_ssl()`

Closes #10054
Fixes #10053

Reported-by: Patrick Monnerat
2022-12-08 14:27:06 +01:00
Stefan Eissing
55807e6c05
tls: backends use connection filters for IO, enabling HTTPS-proxy
- OpenSSL (and compatible)
 - BearSSL
 - gnutls
 - mbedtls
 - rustls
 - schannel
 - secure-transport
 - wolfSSL (v5.0.0 and newer)

 This leaves only the following without HTTPS-proxy support:
 - gskit
 - nss
 - wolfSSL (versions earlier than v5.0.0)

Closes #9962
2022-11-28 13:56:23 +01:00
Patrick Monnerat
eb559c8056
mime: relax easy/mime structures binding
Deprecation and removal of codeset conversion support from the library
have released the strict need for an early binding of mime structures to
an easy handle (https://github.com/curl/curl/commit/2610142).

This constraint currently forces to create the handle before the mime
structure and the latter cannot be attached to another handle once
created (see https://curl.se/mail/lib-2022-08/0027.html).

This commit removes the handle pointers from the mime structures
allowing more flexibility on their use.

When an easy handle is duplicated, bound mime structures must however
still be duplicated too as their components hold send-time dynamic
information.

Closes #9927
2022-11-28 09:09:54 +01:00
Stefan Eissing
f18956d0ca
cfilter: re-add conn as parameter to cfilter setup methods
- `Curl_ssl_get_config()` now returns the first config if no SSL proxy
  filter is active

- socket filter starts connection only on first invocation of its
  connect method

Fixes #9982
Closes #9983
2022-11-26 16:56:46 +01:00
Daniel Gustafsson
2d9fee4504 netware: remove leftover traces
Commit 3b16575ae9 removed support for
building on Novell Netware, but a few leftover traces remained. This
removes the last bits.

Closes: #9966
Reviewed-by: Daniel Stenberg <daniel@haxx.se>
2022-11-23 11:56:52 +01:00
Stefan Eissing
af22c2a546
vtls: localization of state data in filters
- almost all backend calls pass the Curl_cfilter intance instead of
   connectdata+sockindex
 - ssl_connect_data is remove from struct connectdata and made internal
   to vtls
 - ssl_connect_data is allocated in the added filter, kept at cf->ctx

 - added function to let a ssl filter access its ssl_primary_config and
   ssl_config_data this selects the propert subfields in conn and data,
   for filters added as plain or proxy
 - adjusted all backends to use the changed api
 - adjusted all backends to access config data via the exposed
   functions, no longer using conn or data directly

cfilter renames for clear purpose:

 - methods `Curl_conn_*(data, conn, sockindex)` work on the complete
   filter chain at `sockindex` and connection `conn`.
 - methods `Curl_cf_*(cf, ...)` work on a specific Curl_cfilter
   instance.
 - methods `Curl_conn_cf()` work on/with filter instances at a
   connection.
 - rebased and resolved some naming conflicts
 - hostname validation (und session lookup) on SECONDARY use the same
   name as on FIRST (again).

new debug macros and removing connectdata from function signatures where not
needed.

adapting schannel for new Curl_read_plain paramter.

Closes #9919
2022-11-22 14:25:50 +01:00
Daniel Stenberg
bf12c2bed6
lib: remove bad set.opt_no_body assignments
This struct field MUST remain what the application set it to, so that
handle reuse and handle duplication work.

Instead, the request state bit 'no_body' is introduced for code flows
that need to change this in run-time.

Closes #9888
2022-11-11 15:56:05 +01:00
Stefan Eissing
dafdb20a26
lib: connection filters (cfilter) addition to curl:
- general construct/destroy in connectdata
 - default implementations of callback functions
 - connect: cfilters for connect and accept
 - socks: cfilter for socks proxying
 - http_proxy: cfilter for http proxy tunneling
 - vtls: cfilters for primary and proxy ssl
 - change in general handling of data/conn
 - Curl_cfilter_setup() sets up filter chain based on data settings,
   if none are installed by the protocol handler setup
 - Curl_cfilter_connect() boot straps filters into `connected` status,
   used by handlers and multi to reach further stages
 - Curl_cfilter_is_connected() to check if a conn is connected,
   e.g. all filters have done their work
 - Curl_cfilter_get_select_socks() gets the sockets and READ/WRITE
   indicators for multi select to work
 - Curl_cfilter_data_pending() asks filters if the have incoming
   data pending for recv
 - Curl_cfilter_recv()/Curl_cfilter_send are the general callbacks
   installed in conn->recv/conn->send for io handling
 - Curl_cfilter_attach_data()/Curl_cfilter_detach_data() inform filters
   and addition/removal of a `data` from their connection
 - adding vtl functions to prevent use of Curl_ssl globals directly
   in other parts of the code.

Reviewed-by: Daniel Stenberg
Closes #9855
2022-11-11 15:17:51 +01:00
Daniel Stenberg
279834dd45
misc: remove duplicated include files
Closes #9796
2022-10-26 11:27:59 +02:00
Viktor Szakats
c9061f242b
misc: spelling fixes
Found using codespell 2.2.1.

Also delete the redundant protocol designator from an archive.org URL.

Reviewed-by: Daniel Stenberg
Closes #9403
2022-08-31 14:31:01 +00:00
Emil Engler
34886a444f
imap: use ISALNUM() for alphanumeric checks
This commit replaces a self-made character check for alphanumeric
characters within imap_is_bchar() with the ISALNUM() macro, as it is
reduces the size of the code and makes the performance better, due to
ASCII arithmetic.

Closes #9289
2022-08-10 14:12:22 +02:00
max.mehl
ad9bc5976d
copyright: make repository REUSE compliant
Add licensing and copyright information for all files in this repository. This
either happens in the file itself as a comment header or in the file
`.reuse/dep5`.

This commit also adds a Github workflow to check pull requests and adapts
copyright.pl to the changes.

Closes #8869
2022-06-13 09:13:00 +02:00
Daniel Stenberg
6968fb9d54
lib: remove exclamation marks
... from infof() and failf() calls. Make them less attention seeking.

Closes #8713
2022-04-16 11:55:05 +02:00
Daniel Stenberg
7d600ad1c3
urldata: remove conn->bits.user_passwd
The authentication status should be told by the transfer and not the
connection.

Reported-by: John H. Ayad
Fixes #8449
Closes #8451
2022-02-16 10:28:31 +01:00
HenrikHolst
9bc3cebc92
misc: remove strlen for Curl_checkheaders + Curl_checkProxyheaders
Closes #8409
2022-02-10 08:51:06 +01:00
Daniel Stenberg
2610142139
lib: remove support for CURL_DOES_CONVERSIONS
TPF was the only user and support for that was dropped.

Closes #8378
2022-02-04 08:05:35 +01:00
Daniel Stenberg
7a92f865b4
misc: s/e-mail/email
Consistency is king. Following the lead in everything curl.

Closes #8159
2021-12-19 23:48:13 +01:00
Axel Morawietz
49ab21e3d9
imap: display quota information
Show response to "GETQUOTAROOT INBOX" command.

Closes #6973
2021-10-28 21:33:02 +02:00
Patrick Monnerat
3e2c1dcda8
sasl: binary messages
Capabilities of sasl module are extended to exchange messages in binary
as an alternative to base64.

If http authentication flags have been set, those are used as sasl
default preferred mechanisms.

Closes #6930
2021-10-02 23:30:37 +02:00
Patrick Monnerat
8ef147c436
ftp,imap,pop3,smtp: reject STARTTLS server response pipelining
If a server pipelines future responses within the STARTTLS response, the
former are preserved in the pingpong cache across TLS negotiation and
used as responses to the encrypted commands.

This fix detects pipelined STARTTLS responses and rejects them with an
error.

CVE-2021-22947

Bug: https://curl.se/docs/CVE-2021-22947.html
2021-09-13 16:51:31 +02:00
Patrick Monnerat
364f174724
ftp,imap,pop3: do not ignore --ssl-reqd
In imap and pop3, check if TLS is required even when capabilities
request has failed.

In ftp, ignore preauthentication (230 status of server greeting) if TLS
is required.

Bug: https://curl.se/docs/CVE-2021-22946.html

CVE-2021-22946
2021-09-13 16:51:31 +02:00
Daniel Stenberg
fe7b64228d
strerror.h: remove the #include from files not using it 2021-09-09 07:53:42 +02:00
Daniel Stenberg
e7416cfd2b
infof: remove newline from format strings, always append it
- the data needs to be "line-based" anyway since it's also passed to the
  debug callback/application

- it makes infof() work like failf() and consistency is good

- there's an assert that triggers on newlines in the format string

- Also removes a few instances of "..."

- Removes the code that would append "..." to the end of the data *iff*
  it was truncated in infof()

Closes #7357
2021-07-07 22:54:01 +02:00
Daniel Stenberg
a6da296867
vtls: only store TIMER_APPCONNECT for non-proxy connect
Introducing a 'isproxy' argument to the connect function so that it
knows wether to store the time stamp or not.

Reported-by: Yongkang Huang
Fixes #7274
Closes #7274
2021-06-19 23:02:16 +02:00
Daniel Stenberg
0c55fbab45
conn: add 'attach' to protocol handler, make libssh2 use it
The libssh2 backend has SSH session associated with the connection but
the callback context is the easy handle, so when a connection gets
attached to a transfer, the protocol handler now allows for a custom
function to get used to set things up correctly.

Reported-by: Michael O'Farrell
Fixes #6898
Closes #7078
2021-05-17 17:57:50 +02:00
Daniel Stenberg
a9bc819c89
sasl: use 'unsigned short' to store mechanism
... saves a few bytes of struct size in memory and it only uses
10 bits anyway.

Closes #7045
2021-05-12 00:16:52 +02:00
Daniel Stenberg
5f08a5acd5
pingpong: rename the curl_pp_transfer enum to use PP prefix
Using an FTP prefix for PP provided functionality was misleading.
2021-02-04 23:06:03 +01:00
Daniel Stenberg
7173fe98ba
pingpong: remove the 'conn' struct member
... as it's superfluous now when Curl_easy is passed in and we can
derive the connection from that instead and avoid the duplicate copy.

Closes #6525
2021-01-26 10:12:59 +01:00
Patrick Monnerat
ecb13416e3
lib: remove conn->data uses
Closes #6499
2021-01-24 18:15:03 +01:00
Daniel Stenberg
a304051620
lib: more conn->data cleanups
Closes #6479
2021-01-19 09:14:51 +01:00
Patrick Monnerat
bbe3aa9f88
vtls: reduce conn->data use
Closes #6474
2021-01-19 09:14:40 +01:00
Daniel Stenberg
215db086e0
lib: pass in 'struct Curl_easy *' to most functions
... in most cases instead of 'struct connectdata *' but in some cases in
addition to.

- We mostly operate on transfers and not connections.

- We need the transfer handle to log, store data and more. Everything in
  libcurl is driven by a transfer (the CURL * in the public API).

- This work clarifies and separates the transfers from the connections
  better.

- We should avoid "conn->data". Since individual connections can be used
  by many transfers when multiplexing, making sure that conn->data
  points to the current and correct transfer at all times is difficult
  and has been notoriously error-prone over the years. The goal is to
  ultimately remove the conn->data pointer for this reason.

Closes #6425
2021-01-17 23:56:09 +01:00
Daniel Stenberg
aba01da639
failf: remove newline from formatting strings
... as failf adds one itself.

Also: add an assert() to failf() that triggers on a newline in the
format string!

Closes #6365
2020-12-25 23:42:16 +01:00
Daniel Stenberg
a95a6ce6b8
urldata: remove 'void *protop' and create the union 'p'
... to avoid the use of 'void *' for the protocol specific structs done
per transfer.

Closes #6238
2020-11-23 16:16:16 +01:00
Daniel Stenberg
4d2f800677
curl.se: new home
Closes #6172
2020-11-04 23:59:47 +01:00
Daniel Stenberg
c4693adc62
imap: make imap_send use dynbuf for the send buffer management
Reuses the buffer and thereby reduces number of mallocs over a transfer.

Closes #6010
2020-09-25 08:35:01 +02:00
Daniel Stenberg
675eeb1c94
pingpong: use a dynbuf for the *_pp_sendf() function
... reuses the same dynamic buffer instead of doing repeated malloc/free
cycles.

Test case 100 (FTP dir list PASV) does 7 fewer memory allocation calls
after this change in my test setup (132 => 125), curl 7.72.0 needed 140
calls for this.

Test case 103 makes 9 less allocations now (130). Down from 149 in
7.72.0.

Closes #6004
2020-09-23 15:14:09 +02:00
Daniel Stenberg
6434a73984
Curl_handler: add 'family' to each protocol
Makes get_protocol_family() faster and it moves the knowledge about the
"families" to each protocol handler, where it belongs.

Closes #5986
2020-09-21 23:30:16 +02:00
Antarpreet Singh
da5ae38db0
imap: set cselect_bits to CURL_CSELECT_IN initially
... when continuing a transfer from a FETCH response.

When the size of the file was small enough that the entirety of the
transfer happens in a single go and schannel buffers holds the entire
data. However, it wasn't completely read in Curl_pp_readresp since a
line break was found before that could happen. So, by the time we are in
imap_state_fetch_resp - there's data in buffers that needs to be read
via Curl_read but nothing to read from the socket. After we setup a
transfer (Curl_setup_transfer), curl just waits on the socket state to
change - which doesn't happen since no new data ever comes.

Closes #5961
2020-09-14 12:29:44 +02:00
Daniel Stenberg
31e53584db
escape: make the URL decode able to reject only %00 bytes
... or all "control codes" or nothing.

Assisted-by: Nicolas Sterchele
2020-06-25 09:57:18 +02:00
Daniel Stenberg
7414fb25a2
urldata: connect related booleans live in struct ConnectBits
And remove a few unused booleans!

Closes #5461
2020-05-27 08:02:47 +02:00
Daniel Stenberg
d0390a538a
imap: merged two case-branches performing the same action
Fixes warning detected by PVS-Studio
Fixes #4374
2019-09-20 08:07:25 +02:00
Daniel Stenberg
a55faf33d4
cleanup: remove the 'numsocks' argument used in many places
It was used (intended) to pass in the size of the 'socks' array that is
also passed to these functions, but was rarely actually checked/used and
the array is defined to a fixed size of MAX_SOCKSPEREASYHANDLE entries
that should be used instead.

Closes #4169
2019-07-30 23:16:44 +02:00
Marcel Raad
e23c52b329
build: fix Codacy warnings
Reduce variable scopes and remove redundant variable stores.

Closes https://github.com/curl/curl/pull/3975
2019-06-05 20:38:06 +02:00
Daniel Stenberg
8ece8177f1
cleanup: remove FIXME and TODO comments
They serve very little purpose and mostly just add noise. Most of them
have been around for a very long time. I read them all before removing
or rephrasing them.

Ref: #3876
Closes #3883
2019-05-16 09:16:56 +02:00
Daniel Gustafsson
bd91e7f340 imap: Fix typo in comment 2019-05-14 12:38:23 +02:00
Steve Holme
166ace8be0 emailL Added reference to RFC8314 for implicit TLS 2019-04-11 01:19:15 +01:00