http: HEAD response body tolerance

- as reported in #13725, some servers wrongly send body bytes in
  responses to a HEAD request. This used to be tolerated in curl
  8.4 and before and leads to failed transfers in newer versions.
- restore previous behaviour for HTTP/1.1 and HTTP/2:
  * 1.1: do not add 'Transfer-Encoding' writers from HEAD
    responses. RFC 9112 says they do not apply.
  * 2: when the transfer expects 'no_body', to not report stream
    resets as error when all response headers have been received.

Reported-by: Jeroen Ooms
Fixes #13725
Closes #13732
This commit is contained in:
Stefan Eissing 2024-05-21 11:21:14 +02:00 committed by Daniel Stenberg
parent dbd626ab82
commit 5a4769b6d5
No known key found for this signature in database
GPG Key ID: 5CC908FDB71E12C2
2 changed files with 23 additions and 10 deletions

View File

@ -3139,19 +3139,23 @@ CURLcode Curl_http_header(struct Curl_easy *data,
break;
case 't':
case 'T':
v = !k->http_bodyless? HD_VAL(hd, hdlen, "Transfer-Encoding:") : NULL;
/* RFC 9112, ch. 6.1
* "Transfer-Encoding MAY be sent in a response to a HEAD request or
* in a 304 (Not Modified) response (Section 15.4.5 of [HTTP]) to a
* GET request, neither of which includes a message body, to indicate
* that the origin server would have applied a transfer coding to the
* message body if the request had been an unconditional GET."
*
* Read: in these cases the 'Transfer-Encoding' does not apply
* to any data following the response headers. Do not add any decoders.
*/
v = (!k->http_bodyless &&
(data->state.httpreq != HTTPREQ_HEAD) &&
(k->httpcode != 304))?
HD_VAL(hd, hdlen, "Transfer-Encoding:") : NULL;
if(v) {
/* One or more encodings. We check for chunked and/or a compression
algorithm. */
/*
* [RFC 2616, section 3.6.1] A 'chunked' transfer encoding
* means that the server will send a series of "chunks". Each
* chunk starts with line with info (including size of the
* coming block) (terminated with CRLF), then a block of data
* with the previously mentioned size. There can be any amount
* of chunks, and a chunk-data set to zero signals the
* end-of-chunks. */
result = Curl_build_unencoding_stack(data, v, TRUE);
if(result)
return result;

View File

@ -1716,6 +1716,15 @@ static ssize_t http2_handle_stream_close(struct Curl_cfilter *cf,
return -1;
}
else if(stream->error != NGHTTP2_NO_ERROR) {
if(stream->resp_hds_complete && data->req.no_body) {
CURL_TRC_CF(data, cf, "[%d] error after response headers, but we did "
"not want a body anyway, ignore: %s (err %u)",
stream->id, nghttp2_http2_strerror(stream->error),
stream->error);
stream->close_handled = TRUE;
*err = CURLE_OK;
goto out;
}
failf(data, "HTTP/2 stream %u was not closed cleanly: %s (err %u)",
stream->id, nghttp2_http2_strerror(stream->error),
stream->error);