curl/tests/http/Makefile.am

#***************************************************************************
#                                  _   _ ____  _
#  Project                     ___| | | |  _ \| |
#                             / __| | | | |_) | |
#                            | (__| |_| |  _ <| |___
#                             \___|\___/|_| \_\_____|
#
# Copyright (C) Daniel Stenberg, <daniel@haxx.se>, et al.
#
# This software is licensed as described in the file COPYING, which
# you should have received as part of this distribution. The terms
# are also available at https://curl.se/docs/copyright.html.
#
# You may opt to use, copy, modify, merge, publish, distribute and/or sell
# copies of the Software, and permit persons to whom the Software is
# furnished to do so, under the terms of the COPYING file.
#
# This software is distributed on an "AS IS" basis, WITHOUT WARRANTY OF ANY
# KIND, either express or implied.
#
# SPDX-License-Identifier: curl
#
###########################################################################

SUBDIRS = clients

TESTENV =                             \
testenv/__init__.py                   \
testenv/caddy.py                      \
testenv/certs.py                      \
testenv/client.py                     \
testenv/curl.py                       \
testenv/env.py                        \
testenv/httpd.py                      \
testenv/mod_curltest/mod_curltest.c   \
testenv/nghttpx.py                    \
testenv/ports.py                      \
testenv/ws_echo_server.py

EXTRA_DIST =           \
config.ini.in          \
conftest.py            \
requirements.txt       \
scorecard.py           \
test_01_basic.py       \
test_02_download.py    \
test_03_goaway.py      \
test_04_stuttered.py   \
test_05_errors.py      \
test_06_eyeballs.py    \
test_07_upload.py      \
test_08_caddy.py       \
test_09_push.py        \
test_10_proxy.py       \
test_11_unix.py        \
test_12_reuse.py       \
test_13_proxy_auth.py  \
test_14_auth.py        \
test_15_tracing.py     \
test_20_websockets.py  \
$(TESTENV)

clean-local:
	rm -rf *.pyc __pycache__
	rm -rf gen

check: clients

clients:
	@(cd clients; $(MAKE) check)

checksrc:
	cd clients && $(MAKE) checksrc
connections: introduce http/3 happy eyeballs New cfilter HTTP-CONNECT for h3/h2/http1.1 eyeballing. - filter is installed when `--http3` in the tool is used (or the equivalent CURLOPT_ done in the library) - starts a QUIC/HTTP/3 connect right away. Should that not succeed after 100ms (subject to change), a parallel attempt is started for HTTP/2 and HTTP/1.1 via TCP - both attempts are subject to IPv6/IPv4 eyeballing, same as happens for other connections - tie timeout to the ip-version HAPPY_EYEBALLS_TIMEOUT - use a `soft` timeout at half the value. When the soft timeout expires, the HTTPS-CONNECT filter checks if the QUIC filter has received any data from the server. If not, it will start the HTTP/2 attempt. HTTP/3(ngtcp2) improvements. - setting call_data in all cfilter calls similar to http/2 and vtls filters for use in callback where no stream data is available. - returning CURLE_PARTIAL_FILE for prematurely terminated transfers - enabling pytest test_05 for h3 - shifting functionality to "connect" UDP sockets from ngtcp2 implementation into the udp socket cfilter. Because unconnected UDP sockets are weird. For example they error when adding to a pollset. HTTP/3(quiche) improvements. - fixed upload bug in quiche implementation, now passes 251 and pytest - error codes on stream RESET - improved debug logs - handling of DRAIN during connect - limiting pending event queue HTTP/2 cfilter improvements. - use LOG_CF macros for dynamic logging in debug build - fix CURLcode on RST streams to be CURLE_PARTIAL_FILE - enable pytest test_05 for h2 - fix upload pytests and improve parallel transfer performance. GOAWAY handling for ngtcp2/quiche - during connect, when the remote server refuses to accept new connections and closes immediately (so the local conn goes into DRAIN phase), the connection is torn down and a another attempt is made after a short grace period. This is the behaviour observed with nghttpx when we tell it to shut down gracefully. Tested in pytest test_03_02. TLS improvements - ALPN selection for SSL/SSL-PROXY filters in one vtls set of functions, replaces copy of logic in all tls backends. - standardized the infof logging of offered ALPNs - ALPN negotiated: have common function for all backends that sets alpn proprty and connection related things based on the negotiated protocol (or lack thereof). - new tests/tests-httpd/scorecard.py for testing h3/h2 protocol implementation. Invoke: python3 tests/tests-httpd/scorecard.py --help for usage. Improvements on gathering connect statistics and socket access. - new CF_CTRL_CONN_REPORT_STATS cfilter control for having cfilters report connection statistics. This is triggered when the connection has completely connected. - new void Curl_pgrsTimeWas(..) method to report a timer update with a timestamp of when it happend. This allows for updating timers "later", e.g. a connect statistic after full connectivity has been reached. - in case of HTTP eyeballing, the previous changes will update statistics only from the filter chain that "won" the eyeballing. - new cfilter query CF_QUERY_SOCKET for retrieving the socket used by a filter chain. Added methods Curl_conn_cf_get_socket() and Curl_conn_get_socket() for convenient use of this query. - Change VTLS backend to query their sub-filters for the socket when checks during the handshake are made. HTTP/3 documentation on how https eyeballing works. TLS improvements - ALPN selection for SSL/SSL-PROXY filters in one vtls set of functions, replaces copy of logic in all tls backends. - standardized the infof logging of offered ALPNs - ALPN negotiated: have common function for all backends that sets alpn proprty and connection related things based on the negotiated protocol (or lack thereof). Scorecard with Caddy. - configure can be run with `--with-test-caddy=path` to specify which caddy to use for testing - tests/tests-httpd/scorecard.py now measures download speeds with caddy pytest improvements - adding Makfile to clean gen dir - adding nghttpx rundir creation on start - checking httpd version 2.4.55 for test_05 cases where it is needed. Skipping with message if too old. - catch exception when checking for caddy existance on system. Closes #10349 2023-02-02 00:13:12 +08:00			`#***************************************************************************`
			`# _ _ ____ _`
			`# Project ___\| \| \| \| _ \\| \|`
			`# / __\| \| \| \| \|_) \| \|`
			`# \| (__\| \|_\| \| _ <\| \|___`
			`# \___\|\___/\|_\| \_\_____\|`
			`#`
			`# Copyright (C) Daniel Stenberg, <daniel@haxx.se>, et al.`
			`#`
			`# This software is licensed as described in the file COPYING, which`
			`# you should have received as part of this distribution. The terms`
			`# are also available at https://curl.se/docs/copyright.html.`
			`#`
			`# You may opt to use, copy, modify, merge, publish, distribute and/or sell`
			`# copies of the Software, and permit persons to whom the Software is`
			`# furnished to do so, under the terms of the COPYING file.`
			`#`
			`# This software is distributed on an "AS IS" basis, WITHOUT WARRANTY OF ANY`
			`# KIND, either express or implied.`
			`#`
			`# SPDX-License-Identifier: curl`
			`#`
			`###########################################################################`

tests/http: more tests with specific clients - Makefile support for building test specific clients in tests/http/clients - auto-make of clients when invoking pytest - added test_09_02 for server PUSH_PROMISEs using clients/h2-serverpush - added test_02_21 for lib based downloads and pausing/unpausing transfers curl url parser: - added internal method `curl_url_set_authority()` for setting the authority part of a url (used for PUSH_PROMISE) http2: - made logging of PUSH_PROMISE handling nicer Placing python test requirements in requirements.txt files - separate files to base test suite and http tests since use and module lists differ - using the files in the gh workflows websocket test cases, fixes for we and bufq - bufq: account for spare chunks in space calculation - bufq: reset chunks that are skipped empty - ws: correctly encode frames with 126 bytes payload - ws: update frame meta information on first call of collect callback that fills user buffer - test client ws-data: some test/reporting improvements Closes #11006 2023-04-26 18:38:22 +08:00			`SUBDIRS = clients`

dist: make sure the http tests are in the tarball Fixes #12914 Reported-by: Fabian Vogt Closes #12917 2024-02-09 21:15:25 +08:00			`TESTENV = \`
			`testenv/__init__.py \`
			`testenv/caddy.py \`
			`testenv/certs.py \`
			`testenv/client.py \`
			`testenv/curl.py \`
			`testenv/env.py \`
			`testenv/httpd.py \`
			`testenv/mod_curltest/mod_curltest.c \`
			`testenv/nghttpx.py \`
			`testenv/ports.py \`
			`testenv/ws_echo_server.py`

			`EXTRA_DIST = \`
			`config.ini.in \`
			`conftest.py \`
			`requirements.txt \`
			`scorecard.py \`
			`test_01_basic.py \`
			`test_02_download.py \`
			`test_03_goaway.py \`
			`test_04_stuttered.py \`
			`test_05_errors.py \`
			`test_06_eyeballs.py \`
			`test_07_upload.py \`
			`test_08_caddy.py \`
			`test_09_push.py \`
			`test_10_proxy.py \`
			`test_11_unix.py \`
			`test_12_reuse.py \`
			`test_13_proxy_auth.py \`
			`test_14_auth.py \`
			`test_15_tracing.py \`
			`test_20_websockets.py \`
			`$(TESTENV)`

connections: introduce http/3 happy eyeballs New cfilter HTTP-CONNECT for h3/h2/http1.1 eyeballing. - filter is installed when `--http3` in the tool is used (or the equivalent CURLOPT_ done in the library) - starts a QUIC/HTTP/3 connect right away. Should that not succeed after 100ms (subject to change), a parallel attempt is started for HTTP/2 and HTTP/1.1 via TCP - both attempts are subject to IPv6/IPv4 eyeballing, same as happens for other connections - tie timeout to the ip-version HAPPY_EYEBALLS_TIMEOUT - use a `soft` timeout at half the value. When the soft timeout expires, the HTTPS-CONNECT filter checks if the QUIC filter has received any data from the server. If not, it will start the HTTP/2 attempt. HTTP/3(ngtcp2) improvements. - setting call_data in all cfilter calls similar to http/2 and vtls filters for use in callback where no stream data is available. - returning CURLE_PARTIAL_FILE for prematurely terminated transfers - enabling pytest test_05 for h3 - shifting functionality to "connect" UDP sockets from ngtcp2 implementation into the udp socket cfilter. Because unconnected UDP sockets are weird. For example they error when adding to a pollset. HTTP/3(quiche) improvements. - fixed upload bug in quiche implementation, now passes 251 and pytest - error codes on stream RESET - improved debug logs - handling of DRAIN during connect - limiting pending event queue HTTP/2 cfilter improvements. - use LOG_CF macros for dynamic logging in debug build - fix CURLcode on RST streams to be CURLE_PARTIAL_FILE - enable pytest test_05 for h2 - fix upload pytests and improve parallel transfer performance. GOAWAY handling for ngtcp2/quiche - during connect, when the remote server refuses to accept new connections and closes immediately (so the local conn goes into DRAIN phase), the connection is torn down and a another attempt is made after a short grace period. This is the behaviour observed with nghttpx when we tell it to shut down gracefully. Tested in pytest test_03_02. TLS improvements - ALPN selection for SSL/SSL-PROXY filters in one vtls set of functions, replaces copy of logic in all tls backends. - standardized the infof logging of offered ALPNs - ALPN negotiated: have common function for all backends that sets alpn proprty and connection related things based on the negotiated protocol (or lack thereof). - new tests/tests-httpd/scorecard.py for testing h3/h2 protocol implementation. Invoke: python3 tests/tests-httpd/scorecard.py --help for usage. Improvements on gathering connect statistics and socket access. - new CF_CTRL_CONN_REPORT_STATS cfilter control for having cfilters report connection statistics. This is triggered when the connection has completely connected. - new void Curl_pgrsTimeWas(..) method to report a timer update with a timestamp of when it happend. This allows for updating timers "later", e.g. a connect statistic after full connectivity has been reached. - in case of HTTP eyeballing, the previous changes will update statistics only from the filter chain that "won" the eyeballing. - new cfilter query CF_QUERY_SOCKET for retrieving the socket used by a filter chain. Added methods Curl_conn_cf_get_socket() and Curl_conn_get_socket() for convenient use of this query. - Change VTLS backend to query their sub-filters for the socket when checks during the handshake are made. HTTP/3 documentation on how https eyeballing works. TLS improvements - ALPN selection for SSL/SSL-PROXY filters in one vtls set of functions, replaces copy of logic in all tls backends. - standardized the infof logging of offered ALPNs - ALPN negotiated: have common function for all backends that sets alpn proprty and connection related things based on the negotiated protocol (or lack thereof). Scorecard with Caddy. - configure can be run with `--with-test-caddy=path` to specify which caddy to use for testing - tests/tests-httpd/scorecard.py now measures download speeds with caddy pytest improvements - adding Makfile to clean gen dir - adding nghttpx rundir creation on start - checking httpd version 2.4.55 for test_05 cases where it is needed. Skipping with message if too old. - catch exception when checking for caddy existance on system. Closes #10349 2023-02-02 00:13:12 +08:00			`clean-local:`
			`rm -rf *.pyc __pycache__`
			`rm -rf gen`
tests/http: more tests with specific clients - Makefile support for building test specific clients in tests/http/clients - auto-make of clients when invoking pytest - added test_09_02 for server PUSH_PROMISEs using clients/h2-serverpush - added test_02_21 for lib based downloads and pausing/unpausing transfers curl url parser: - added internal method `curl_url_set_authority()` for setting the authority part of a url (used for PUSH_PROMISE) http2: - made logging of PUSH_PROMISE handling nicer Placing python test requirements in requirements.txt files - separate files to base test suite and http tests since use and module lists differ - using the files in the gh workflows websocket test cases, fixes for we and bufq - bufq: account for spare chunks in space calculation - bufq: reset chunks that are skipped empty - ws: correctly encode frames with 126 bytes payload - ws: update frame meta information on first call of collect callback that fills user buffer - test client ws-data: some test/reporting improvements Closes #11006 2023-04-26 18:38:22 +08:00
			`check: clients`

			`clients:`
			`@(cd clients; $(MAKE) check)`

			`checksrc:`
			`cd clients && $(MAKE) checksrc`