Commit Graph

27 Commits

Author SHA1 Message Date
Dr. David von Oheimb
cdaf072f90 HTTP client: Fix cleanup of TLS BIO via 'bio_update_fn' callback function
Make app_http_tls_cb() tidy up on disconnect the SSL BIO it pushes on connect.
Make OSSL_HTTP_close() respect this.

Reviewed-by: Paul Dale <pauli@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/17318)
2021-12-22 12:24:24 +01:00
Dr. David von Oheimb
119f8145c3 OSSL_HTTP_open(): clarify doc of 'server' arg and its use of BIO_new_connect()
Reviewed-by: Paul Dale <pauli@openssl.org>
Reviewed-by: Tomas Mraz <tomas@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/17186)
2021-12-07 11:26:49 +01:00
Dr. David von Oheimb
59b6b5a94f OSSL_HTTP_open(): Complete documentation of checks for server and proxy args
Reviewed-by: Paul Dale <pauli@openssl.org>
Reviewed-by: Tomas Mraz <tomas@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/17186)
2021-12-07 11:26:49 +01:00
Dr. David von Oheimb
266383b44c OSSL_HTTP_set1_request(): Fix check for presence of port option and its documentation
For HTTP (not HTTPS) with proxy, server must be given, port is optional

Reviewed-by: Paul Dale <pauli@openssl.org>
Reviewed-by: Tomas Mraz <tomas@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/17186)
2021-12-07 11:26:49 +01:00
Dr. David von Oheimb
2080134ee9 OSSL_HTTP_transfer.pod: Some clarifications on the BIO connect/disconnect callback function
Reviewed-by: Paul Dale <pauli@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/17160)
2021-12-03 11:57:47 +01:00
Dr. David von Oheimb
4ee464cf8e OSSL_HTTP_transfer.pod: Fix omission documenting the 'ok' parameter of OSSL_HTTP_close()
Reviewed-by: Paul Dale <pauli@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/17160)
2021-12-03 11:57:47 +01:00
Dr. David von Oheimb
a26c089ba3 OSSL_HTTP_transfer.pod: clarify that resulting BIO must be freed
Reviewed-by: Tomas Mraz <tomas@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/16888)
2021-10-23 12:55:03 +02:00
Dr. David von Oheimb
8ccbf00d17 Rename OSSL_HTTP_set_request() to OSSL_HTTP_set1_request() for clarity
Reviewed-by: Tomas Mraz <tomas@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/15697)
2021-06-11 14:41:20 +02:00
Dr. David von Oheimb
647a5dbf10 Add OSSL_ prefix to HTTP_DEFAULT_MAX_{LINE_LENGTH,RESP_LEN}
Reviewed-by: Tomas Mraz <tomas@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/15053)
2021-05-14 19:24:42 +02:00
Dr. David von Oheimb
be799eb7a3 HTTP client: Allow streaming of response data (with possibly indefinite length)
Also clean up max_resp_len and add OSSL_HTTP_REQ_CTX_get_resp_len().

Reviewed-by: Tomas Mraz <tomas@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/15053)
2021-05-14 19:24:42 +02:00
Dr. David von Oheimb
8b5ca5111e HTTP client: Allow streaming of request data (for POST method)
Also clean up OSSL_HTTP_REQ_CTX_nbio() states and make it more efficient.

Reviewed-by: Tomas Mraz <tomas@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/15053)
2021-05-14 19:24:42 +02:00
Dr. David von Oheimb
829902879e HTTP client API: Generalize to arbitrary request and response contents
Reviewed-by: Tomas Mraz <tomas@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/15053)
2021-05-14 19:24:42 +02:00
Dr. David von Oheimb
19f97fe6f1 HTTP: Implement persistent connections (keep-alive)
Both at API and at CLI level (for the CMP app only, so far)
there is a new parameter/option: keep_alive.
* 0 means HTTP connections are not kept open after
receiving a response, which is the default behavior for HTTP 1.0.
* 1 means that persistent connections are requested.
* 2 means that persistent connections are required, i.e.,
in case the server does not grant them an error occurs.

For the CMP app the default value is 1, which means preferring to keep
the connection open. For all other internal uses of the HTTP client
(fetching an OCSP response, a cert, or a CRL) it does not matter
because these operations just take one round trip.

If the client application requested or required a persistent connection
and this was granted by the server, it can keep the OSSL_HTTP_REQ_CTX *
as long as it wants to send further requests and OSSL_HTTP_is_alive()
returns nonzero,
else it should call OSSL_HTTP_REQ_CTX_free() or OSSL_HTTP_close().
In case the client application keeps the OSSL_HTTP_REQ_CTX *
but the connection then dies for any reason at the server side, it will
notice this obtaining an I/O error when trying to send the next request.

This requires extending the HTTP header parsing and
rearranging the high-level HTTP client API. In particular:
* Split the monolithic OSSL_HTTP_transfer() into OSSL_HTTP_open(),
  OSSL_HTTP_set_request(), a lean OSSL_HTTP_transfer(), and OSSL_HTTP_close().
* Split the timeout functionality accordingly and improve default behavior.
* Extract part of OSSL_HTTP_REQ_CTX_new() to OSSL_HTTP_REQ_CTX_set_expected().
* Extend struct ossl_http_req_ctx_st accordingly.

Use the new feature for the CMP client, which requires extending
related transaction management of CMP client and test server.

Update the documentation and extend the tests accordingly.

Reviewed-by: Tomas Mraz <tomas@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/15053)
2021-05-14 19:24:42 +02:00
Dr. David von Oheimb
8f965908a5 HTTP client: Minimal changes that include the improved API
This is a minimal version of pull request #15053 including all the
proposed improvements to the HTTP client API and its documentation
but only those code adaptations strictly needed for it.

The proposed new features include
* support for persistent connections (keep-alive),
* generalization to arbitrary request and response types, and
* support for streaming BIOs for request and response data.

The related API changes include:
* Split the monolithic OSSL_HTTP_transfer() into OSSL_HTTP_open(),
  OSSL_HTTP_set_request(), a lean OSSL_HTTP_transfer(), and OSSL_HTTP_close().
* Split the timeout functionality accordingly and improve default behavior.
* Extract part of OSSL_HTTP_REQ_CTX_new() to OSSL_HTTP_REQ_CTX_set_expected().

Reviewed-by: Tomas Mraz <tomas@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/15147)
2021-05-12 15:11:51 +02:00
Dr. David von Oheimb
79a2bccdb0 HTTP client: Correct the use of optional proxy URL and its documentation
Reviewed-by: Tomas Mraz <tomas@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/15104)
2021-05-04 18:27:57 +02:00
Dr. David von Oheimb
7932982b88 OSSL_HTTP_parse_url(): Handle any userinfo, query, and fragment components
Now handle [http[s]://][userinfo@]host[:port][/path][?query][#frag]
by optionally providing any userinfo, query, and frag components.

All usages of this function, which are client-only,
silently ignore userinfo and frag components,
while the query component is taken as part of the path.
Update and extend the unit tests and all affected documentation.
Document and deprecat OCSP_parse_url().

Fixes an issue that came up when discussing FR #14001.

Reviewed-by: Richard Levitte <levitte@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/14009)
2021-03-01 10:30:43 +01:00
Dr. David von Oheimb
6aab42c390 OSSL_HTTP_REQ_CTX.pod and OSSL_HTTP_transfer.pod: various improvements
Reviewed-by: Paul Dale <pauli@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/13960)
2021-02-02 07:54:37 +01:00
Dr. David von Oheimb
d337af1891 HTTP: Fix mistakes and unclarities on maxline and max_resp_len params
Also rename internal structure fields iobuf(len) to readbuf(len) for clarity

Reviewed-by: Paul Dale <pauli@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/13960)
2021-02-02 07:54:37 +01:00
Richard Levitte
4333b89f50 Update copyright year
Reviewed-by: Tomas Mraz <tomas@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/13999)
2021-01-28 13:54:57 +01:00
Richard Levitte
ad2cc1a08e Make the OSSL_HTTP manual conform with man-pages(7)
Details from man-pages(7) that are used:

    Formatting conventions for manual pages describing functions

        ...
        Variable names should, like argument names, be specified in italics.
        ...

    Formatting conventions (general)

        ...
        Special macros, which are usually in uppercase, are in bold.
        Exception: don't boldface NULL.
        ...

Reviewed-by: Tomas Mraz <tmraz@fedoraproject.org>
(Merged from https://github.com/openssl/openssl/pull/13847)
2021-01-13 23:33:16 +01:00
Dr. David von Oheimb
d7fcee3b3b OSSL_HTTP_parse_url(): add optional port number return parameter and strengthen documentation
Reviewed-by: Shane Lontis <shane.lontis@oracle.com>
(Merged from https://github.com/openssl/openssl/pull/12786)
2020-09-08 15:36:24 +02:00
Sebastian Andrzej Siewior
bb361a27ba DOC: Spelling fixes
Signed-off-by: Sebastian Andrzej Siewior <sebastian@breakpoint.cc>

Reviewed-by: Paul Dale <paul.dale@oracle.com>
Reviewed-by: Dmitry Belyavskiy <beldmit@gmail.com>
(Merged from https://github.com/openssl/openssl/pull/12172)
2020-06-18 10:10:48 +03:00
Dr. David von Oheimb
e98c7350bf Improve BIO_socket_wait(), BIO_wait(), BIO_connect_retry(), and their docs
Add/extend range check for 'fd' argument of BIO_socket_wait() and bio_wait()
Correct nap time calculations in bio_wait(), thus correcting also BIO_wait()
Update a type cast from 'unsigned long' to 'unsigned int'
Extend the comments and documentation of BIO_wait()

Rename BIO_connect_retry() to BIO_do_connect_retry()
Make its 'timeout' argument < 0 lead to BIO_do_connect() tried only once
Add optional 'nap_milliseconds' parameter determining the polling granularity
Correct and generalize the retry case checking
Extend the comments and documentation of BIO_do_connect_retry()

Reviewed-by: Bernd Edlinger <bernd.edlinger@hotmail.de>
Reviewed-by: Matt Caswell <matt@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/11986)
2020-06-16 18:51:39 +02:00
Dr. David von Oheimb
9253f8346a Constify 'req' parameter of OSSL_HTTP_post_asn1()
Reviewed-by: Matt Caswell <matt@openssl.org>
Reviewed-by: Viktor Dukhovni <viktor@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/11736)
2020-05-09 16:57:08 +02:00
Dr. David von Oheimb
4b1fe471ac HTTP client: make server/proxy and port params more consistent; minor other improvements
Reviewed-by: Matt Caswell <matt@openssl.org>
Reviewed-by: David von Oheimb <david.von.oheimb@siemens.com>
(Merged from https://github.com/openssl/openssl/pull/11404)
2020-04-02 18:31:06 +02:00
Dr. David von Oheimb
afe554c2d2 Chunk 10 of CMP contribution to OpenSSL: CMP http client and related tests
Also improve the generic HTTP client w.r.t. proxy and no_proxy options.

    Certificate Management Protocol (CMP, RFC 4210) extension to OpenSSL
    Also includes CRMF (RFC 4211) and HTTP transfer (RFC 6712).
    Adds the CMP and CRMF API to libcrypto and the "cmp" app to the CLI.
    Adds extensive documentation and tests.

Reviewed-by: Matt Caswell <matt@openssl.org>
Reviewed-by: David von Oheimb <david.von.oheimb@siemens.com>
(Merged from https://github.com/openssl/openssl/pull/11404)
2020-04-02 18:17:00 +02:00
Dr. David von Oheimb
29f178bddf Generalize the HTTP client so far implemented mostly in crypto/ocsp/ocsp_ht.c
The new client has become an independent libcrpyto module in crypto/http/ and
* can handle any types of requests and responses (ASN.1-encoded and plain)
* does not include potentially busy loops when waiting for responses but
* makes use of a new timeout mechanism integrated with socket-based BIO
* supports the use of HTTP proxies and TLS, including HTTPS over proxies
* supports HTTP redirection via codes 301 and 302 for GET requests
* returns more useful diagnostics in various error situations
Also adapts - and strongly simplifies - hitherto uses of HTTP in crypto/ocsp/,
crypto/x509/x_all.c, apps/lib/apps.c, and apps/{ocsp,s_client,s_server}.c

Reviewed-by: Matt Caswell <matt@openssl.org>
Reviewed-by: David von Oheimb <david.von.oheimb@siemens.com>
(Merged from https://github.com/openssl/openssl/pull/10667)
2020-02-10 16:49:37 +01:00