Commit Graph

86 Commits

Author SHA1 Message Date
Dr. David von Oheimb
7d5019c15a http_client.c: fix comment and documentation of the memory BIOs used
Reviewed-by: Tomas Mraz <tomas@openssl.org>
Reviewed-by: Paul Dale <pauli@openssl.org>
Reviewed-by: Hugo Landau <hlandau@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/18674)
2022-07-06 08:21:53 +01:00
Dr. David von Oheimb
059a4ad099 OSSL_HTTP_REQ_CTX_nbio(): fix copy&paste glitch calling BIO_should_retry(rctx-rbio)
Reviewed-by: Tomas Mraz <tomas@openssl.org>
Reviewed-by: Paul Dale <pauli@openssl.org>
Reviewed-by: Hugo Landau <hlandau@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/18674)
2022-07-06 08:21:53 +01:00
Dr. David von Oheimb
35750cb9af OSSL_HTTP_open(): improve use of use_ssl and its documentation
Reviewed-by: Tomas Mraz <tomas@openssl.org>
Reviewed-by: Paul Dale <pauli@openssl.org>
Reviewed-by: Hugo Landau <hlandau@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/18674)
2022-07-06 08:21:53 +01:00
Jiasheng Jiang
816d6e578c crypto/http/http_client.c: Add the check for OPENSSL_strdup
As the potential failure of the OPENSSL_strdup(), it should
be better to check the return value and return error if fails.

Signed-off-by: Jiasheng Jiang <jiasheng@iscas.ac.cn>

Reviewed-by: Paul Dale <pauli@openssl.org>
Reviewed-by: Tomas Mraz <tomas@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/18624)
2022-06-23 12:35:09 +02:00
Dr. David von Oheimb
e8fdb06035 http_client.c: Dump response on error when tracing is enabled
Reviewed-by: Tomas Mraz <tomas@openssl.org>
Reviewed-by: Dmitry Belyavskiy <beldmit@gmail.com>
Reviewed-by: David von Oheimb <david.von.oheimb@siemens.com>
(Merged from https://github.com/openssl/openssl/pull/18386)
2022-05-30 22:43:44 +02:00
Matt Caswell
fecb3aae22 Update copyright year
Reviewed-by: Tomas Mraz <tomas@openssl.org>
Release: yes
2022-05-03 13:34:51 +01:00
Dr. David von Oheimb
e3477d3e5c http_client.c: check expected content type only if HTTP status code is 200 (OK)
Reviewed-by: Paul Dale <pauli@openssl.org>
Reviewed-by: Tomas Mraz <tomas@openssl.org>
Reviewed-by: David von Oheimb <david.von.oheimb@siemens.com>
(Merged from https://github.com/openssl/openssl/pull/18204)
2022-05-02 08:22:31 +02:00
Dmitry Belyavskiy
fba140c735 str[n]casecmp => OPENSSL_strncasecmp
Reviewed-by: Tim Hudson <tjh@openssl.org>
Reviewed-by: Tomas Mraz <tomas@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/18069)
2022-04-22 11:34:41 +02:00
Richard Levitte
d5f9166bac Move e_os.h to include/internal
Including e_os.h with a path from a header file doesn't work well on
certain exotic platform.  It simply fails to build.

Since we don't seem to be able to stop ourselves, the better move is
to move e_os.h to an include directory that's part of the inclusion
path given to the compiler.

Reviewed-by: Paul Dale <pauli@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/17641)
2022-02-05 05:31:09 +01:00
Dr. David von Oheimb
068549f8db HTTP client: Work around HTTPS proxy use bug due to callback design flaw
See discussion in #17088, where the real solution was postponed to 4.0.

This preliminarily fixes the issue that the HTTP(S) proxy environment vars
were neglected when determining whether a proxy should be used for HTTPS.

Reviewed-by: Tomas Mraz <tomas@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/17310)
2022-01-04 15:05:32 +01:00
Dr. David von Oheimb
cdaf072f90 HTTP client: Fix cleanup of TLS BIO via 'bio_update_fn' callback function
Make app_http_tls_cb() tidy up on disconnect the SSL BIO it pushes on connect.
Make OSSL_HTTP_close() respect this.

Reviewed-by: Paul Dale <pauli@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/17318)
2021-12-22 12:24:24 +01:00
Dr. David von Oheimb
606c79e29b HTTP client: Work around the 'gets' method not being supported by SSL BIOs
It turned out that loading non-ASN.1 contents using the HTTP client
fails over TLS because SSL BIOs do not support the gets method.

This PR provides a workaround by using the less efficient BIO_get_line() function
in case BIO_gets() returns -2, which means that it is not supported by the BIO.

Reviewed-by: Tomas Mraz <tomas@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/17317)
2021-12-21 13:04:14 +01:00
Dr. David von Oheimb
2490d10d5c OSSL_HTTP_proxy_connect(): Fix glitch in response HTTP header parsing
Fixes #17247

Reviewed-by: Paul Dale <pauli@openssl.org>
Reviewed-by: Tomas Mraz <tomas@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/17250)
2021-12-13 12:13:30 +01:00
Dr. David von Oheimb
f0d5a3b6ea OSSL_HTTP_get(): Fix timeout handling on redirection
Reviewed-by: Paul Dale <pauli@openssl.org>
Reviewed-by: Tomas Mraz <tomas@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/17190)
2021-12-09 18:10:07 +01:00
Dr. David von Oheimb
266383b44c OSSL_HTTP_set1_request(): Fix check for presence of port option and its documentation
For HTTP (not HTTPS) with proxy, server must be given, port is optional

Reviewed-by: Paul Dale <pauli@openssl.org>
Reviewed-by: Tomas Mraz <tomas@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/17186)
2021-12-07 11:26:49 +01:00
Dr. David von Oheimb
38288f424f OSSL_HTTP_REQ_CTX_nbio(): Fix parsing of responses with status code != 200
This way keep-alive is not (needlessly) cancelled on error.

Reviewed-by: Tomas Mraz <tomas@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/17171)
2021-12-03 12:21:12 +01:00
Dr. David von Oheimb
e2b7dc353b parse_http_line1(): Fix diagnostic output on error and return code
Reviewed-by: Tomas Mraz <tomas@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/17171)
2021-12-03 12:21:12 +01:00
Dr. David von Oheimb
af16097feb Move more general parts of internal/cryptlib.h to new internal/common.h
Reviewed-by: Paul Dale <pauli@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/15847)
2021-11-17 15:48:37 +01:00
Dr. David von Oheimb
2ff286c26c Add and use HAS_PREFIX() and CHECK_AND_SKIP_PREFIX() for checking if string has literal prefix
Reviewed-by: Paul Dale <pauli@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/15847)
2021-11-17 15:48:34 +01:00
Dr. David von Oheimb
981a5b7ce3 OSSL_HTTP_open(): Fix memory leak on TLS connect failure via proxy
Reviewed-by: Tomas Mraz <tomas@openssl.org>
Reviewed-by: Paul Dale <pauli@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/16119)
2021-07-22 10:14:47 +02:00
Dr. David von Oheimb
6a1f9cdc0c Improve doc of OSSL_HTTP_REQ_CTX_set_expected() on timeout param < 0
Reviewed-by: Paul Dale <pauli@openssl.org>
Reviewed-by: Tomas Mraz <tomas@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/16029)
2021-07-11 17:06:12 +02:00
Dr. David von Oheimb
47bb597b4f http_client.c: fix HTTP_VERSION_STR_LEN and make it more efficient
Reviewed-by: Tomas Mraz <tomas@openssl.org>
Reviewed-by: Paul Dale <pauli@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/15796)
2021-06-18 14:29:19 +02:00
Dr. David von Oheimb
765860a3ce http_client.c: make HTTP_LINE1_MINLEN more efficient
Reviewed-by: Tomas Mraz <tomas@openssl.org>
Reviewed-by: Paul Dale <pauli@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/15796)
2021-06-18 14:29:19 +02:00
Dr. David von Oheimb
b6fec9658b http_client.c: make prefix checking more readable and more efficient
Reviewed-by: Tomas Mraz <tomas@openssl.org>
Reviewed-by: Paul Dale <pauli@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/15796)
2021-06-18 14:29:19 +02:00
Dr. David von Oheimb
552aeaef0d http_client.c: fix error reporting (a char was missing; improve style)
Reviewed-by: Tomas Mraz <tomas@openssl.org>
Reviewed-by: Paul Dale <pauli@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/15796)
2021-06-18 14:29:19 +02:00
Dr. David von Oheimb
8df299d63d http_client.c: fix OSSL_HTTP_proxy_connect() for HTTPS proxy use
Reviewed-by: Tomas Mraz <tomas@openssl.org>
Reviewed-by: Paul Dale <pauli@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/15796)
2021-06-18 14:29:13 +02:00
Dr. David von Oheimb
ab9d67efa4 HTTP client: fix use of OSSL_HTTP_adapt_proxy(), which is needed also in cmp.c
For this reason, export this function, which allows removing http_local.h

Reviewed-by: Tomas Mraz <tomas@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/15764)
2021-06-16 14:36:01 +01:00
Dr. David von Oheimb
8ccbf00d17 Rename OSSL_HTTP_set_request() to OSSL_HTTP_set1_request() for clarity
Reviewed-by: Tomas Mraz <tomas@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/15697)
2021-06-11 14:41:20 +02:00
Dr. David von Oheimb
95c0b295de HTTP client: Fix GET request handling when rctx is reused (keep-alive)
This also updates the documentation of OSSL_HTTP_REQ_CTX_set1_req().

Reviewed-by: Tomas Mraz <tomas@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/15697)
2021-06-11 14:41:20 +02:00
Tomas Mraz
de5a0198b2 Fix use after free in OSSL_HTTP_REQ_CTX_set1_req()
Fixes #15647

Reviewed-by: David von Oheimb <david.von.oheimb@siemens.com>
Reviewed-by: Paul Dale <pauli@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/15648)
2021-06-10 18:27:15 +10:00
Pauli
60e91cc409 http: remove TODOs
Reviewed-by: Tim Hudson <tjh@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/15539)
2021-06-02 16:30:15 +10:00
Pauli
b93f6c2db9 err: rename err_load_xxx_strings_int functions
The new names are ossl_err_load_xxx_strings.

Reviewed-by: Tomas Mraz <tomas@openssl.org>
Reviewed-by: Richard Levitte <levitte@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/15446)
2021-05-26 13:01:47 +10:00
Dr. David von Oheimb
647a5dbf10 Add OSSL_ prefix to HTTP_DEFAULT_MAX_{LINE_LENGTH,RESP_LEN}
Reviewed-by: Tomas Mraz <tomas@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/15053)
2021-05-14 19:24:42 +02:00
Dr. David von Oheimb
e2c38c1a4e http_client.c: Rename internal fields and functions for consistency
Reviewed-by: Tomas Mraz <tomas@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/15053)
2021-05-14 19:24:42 +02:00
Dr. David von Oheimb
be799eb7a3 HTTP client: Allow streaming of response data (with possibly indefinite length)
Also clean up max_resp_len and add OSSL_HTTP_REQ_CTX_get_resp_len().

Reviewed-by: Tomas Mraz <tomas@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/15053)
2021-05-14 19:24:42 +02:00
Dr. David von Oheimb
8b5ca5111e HTTP client: Allow streaming of request data (for POST method)
Also clean up OSSL_HTTP_REQ_CTX_nbio() states and make it more efficient.

Reviewed-by: Tomas Mraz <tomas@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/15053)
2021-05-14 19:24:42 +02:00
Dr. David von Oheimb
829902879e HTTP client API: Generalize to arbitrary request and response contents
Reviewed-by: Tomas Mraz <tomas@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/15053)
2021-05-14 19:24:42 +02:00
Dr. David von Oheimb
22fe2b1299 OSSL_HTTP_transfer(): Fix error reporting in case rctx->server is NULL
Also improve doc of OSSL_parse_url() and OSSL_HTTP_parse_url().

Reviewed-by: Tomas Mraz <tomas@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/15053)
2021-05-14 19:24:42 +02:00
Dr. David von Oheimb
8801240bc5 OSSL_HTTP_get(): Do not close connection if redirect to same server
Reviewed-by: Tomas Mraz <tomas@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/15053)
2021-05-14 19:24:42 +02:00
Dr. David von Oheimb
19f97fe6f1 HTTP: Implement persistent connections (keep-alive)
Both at API and at CLI level (for the CMP app only, so far)
there is a new parameter/option: keep_alive.
* 0 means HTTP connections are not kept open after
receiving a response, which is the default behavior for HTTP 1.0.
* 1 means that persistent connections are requested.
* 2 means that persistent connections are required, i.e.,
in case the server does not grant them an error occurs.

For the CMP app the default value is 1, which means preferring to keep
the connection open. For all other internal uses of the HTTP client
(fetching an OCSP response, a cert, or a CRL) it does not matter
because these operations just take one round trip.

If the client application requested or required a persistent connection
and this was granted by the server, it can keep the OSSL_HTTP_REQ_CTX *
as long as it wants to send further requests and OSSL_HTTP_is_alive()
returns nonzero,
else it should call OSSL_HTTP_REQ_CTX_free() or OSSL_HTTP_close().
In case the client application keeps the OSSL_HTTP_REQ_CTX *
but the connection then dies for any reason at the server side, it will
notice this obtaining an I/O error when trying to send the next request.

This requires extending the HTTP header parsing and
rearranging the high-level HTTP client API. In particular:
* Split the monolithic OSSL_HTTP_transfer() into OSSL_HTTP_open(),
  OSSL_HTTP_set_request(), a lean OSSL_HTTP_transfer(), and OSSL_HTTP_close().
* Split the timeout functionality accordingly and improve default behavior.
* Extract part of OSSL_HTTP_REQ_CTX_new() to OSSL_HTTP_REQ_CTX_set_expected().
* Extend struct ossl_http_req_ctx_st accordingly.

Use the new feature for the CMP client, which requires extending
related transaction management of CMP client and test server.

Update the documentation and extend the tests accordingly.

Reviewed-by: Tomas Mraz <tomas@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/15053)
2021-05-14 19:24:42 +02:00
Dr. David von Oheimb
19a39b29e8 OSSL_HTTP_REQ_CTX_add1_headers(): Fix use with host == NULL (relative URLs)
Reviewed-by: Tomas Mraz <tomas@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/15053)
2021-05-14 19:24:42 +02:00
Dr. David von Oheimb
68bb06f778 HTTP client: Rename 'maxline' parameter to 'buf_size' for clarity
Reviewed-by: Tomas Mraz <tomas@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/15053)
2021-05-14 19:24:42 +02:00
Dr. David von Oheimb
c4005c8b84 http_client.c: Fix inconsistency w.r.t. type of max_resp_len
Reviewed-by: Tomas Mraz <tomas@openssl.org>
Reviewed-by: Richard Levitte <levitte@openssl.org>
Reviewed-by: Paul Dale <pauli@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/15251)
2021-05-13 19:39:40 +02:00
Dr. David von Oheimb
8f965908a5 HTTP client: Minimal changes that include the improved API
This is a minimal version of pull request #15053 including all the
proposed improvements to the HTTP client API and its documentation
but only those code adaptations strictly needed for it.

The proposed new features include
* support for persistent connections (keep-alive),
* generalization to arbitrary request and response types, and
* support for streaming BIOs for request and response data.

The related API changes include:
* Split the monolithic OSSL_HTTP_transfer() into OSSL_HTTP_open(),
  OSSL_HTTP_set_request(), a lean OSSL_HTTP_transfer(), and OSSL_HTTP_close().
* Split the timeout functionality accordingly and improve default behavior.
* Extract part of OSSL_HTTP_REQ_CTX_new() to OSSL_HTTP_REQ_CTX_set_expected().

Reviewed-by: Tomas Mraz <tomas@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/15147)
2021-05-12 15:11:51 +02:00
Dr. David von Oheimb
79a2bccdb0 HTTP client: Correct the use of optional proxy URL and its documentation
Reviewed-by: Tomas Mraz <tomas@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/15104)
2021-05-04 18:27:57 +02:00
Tomas Mraz
bf6aeeb481 http/http_lib.c: Include stdio.h for sscanf()
Reviewed-by: Paul Dale <pauli@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/14953)
2021-04-22 17:45:13 +02:00
Dr. David von Oheimb
1e6174b1b5 OSSL_HTTP_REQ_CTX_transfer(): improve distinction of send error vs. receive error
Reviewed-by: Tomas Mraz <tomas@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/14630)
2021-03-31 19:53:05 +02:00
Dr. David von Oheimb
2318379119 OSSL_parse_url(): Improve handling of IPv6 addresses
Reviewed-by: Tomas Mraz <tomas@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/14630)
2021-03-31 19:53:05 +02:00
Dr. David von Oheimb
e1428c62a1 http_client.c: Prevent spurious error queue entry on NULL mem argument
Reviewed-by: Tomas Mraz <tomas@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/14630)
2021-03-31 19:53:05 +02:00
Dr. David von Oheimb
534725fd43 HTTP: Fix method_POST param by moving it to OSSL_HTTP_REQ_CTX_set_request_line()
Reviewed-by: Tomas Mraz <tomas@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/14699)
2021-03-31 19:51:52 +02:00