Commit Graph

20 Commits

Author SHA1 Message Date
Daniel Stenberg
0efe8b215c
spellcheck: adapt to backslashed minuses
As the curl.1 has more backslashed minus, the cleanup sed lines xneed to
adapt.

Adjusted some docs slighly.

Follow-up to 439ff2052e

Closes #11663
2023-08-12 11:10:21 +02:00
Daniel Stenberg
6b11d9d86b
CI/spellcheck: build curl.1 and spellcheck it
Added acceptable words

Closes #11562
2023-08-01 14:12:03 +02:00
Disyer
4f9c20d49f
wolfssl: support loading system CA certificates
Closes #11452
2023-07-31 08:27:50 +02:00
Daniel Stenberg
4337242345
libcurl-ws.3. WebSocket API overview
Closes #11314
2023-06-14 13:14:57 +02:00
Daniel Stenberg
259ee6defc
curl_pushheader_byname/bynum.3: document in their own man pages
These two functions were added in 7.44.0 when CURLMOPT_PUSHFUNCTION was
introduced but always lived a life in the shadows, embedded in the
CURLMOPT_PUSHFUNCTION man page. Until now.

It makes better sense and gives more visibility to document them in
their own stand-alone man pages.

Closes #11286
2023-06-09 14:25:47 +02:00
Dan Fandrich
e7a021e135 tests: use %LOGDIR to refer to the log directory
This will allow it be set dynamically.

Ref: #10818
2023-03-30 09:53:57 -07:00
Stefan Eissing
6d6404aca0
docs: add documentation for bufq
Closes #10869
2023-03-30 17:13:53 +02:00
Daniel Stenberg
843a72b456
spellcheck.words: unify the AWS-LC spelling
Follow-up to 34ef4fab22

Closes #10867
2023-03-30 11:40:00 +02:00
Jim King
34ef4fab22
openssl: interop with AWS-LC
* Configure changes to detect AWS-LC
* CMakeLists.txt changes to detect AWS-LC
* Compile-time branches needed to support AWS-LC
* Correctly set OSSL_VERSION and report AWS-LC release number
* GitHub Actions script to build with autoconf and cmake against AWS-LC

AWS-LC is a BoringSSL/OpenSSL derivative
For more information see https://github.com/awslabs/aws-lc/

Closes #10320
2023-03-30 10:56:14 +02:00
Stefan Eissing
671158242d
connections: introduce http/3 happy eyeballs
New cfilter HTTP-CONNECT for h3/h2/http1.1 eyeballing.
- filter is installed when `--http3` in the tool is used (or
  the equivalent CURLOPT_ done in the library)
- starts a QUIC/HTTP/3 connect right away. Should that not
  succeed after 100ms (subject to change), a parallel attempt
  is started for HTTP/2 and HTTP/1.1 via TCP
- both attempts are subject to IPv6/IPv4 eyeballing, same
  as happens for other connections
- tie timeout to the ip-version HAPPY_EYEBALLS_TIMEOUT
- use a `soft` timeout at half the value. When the soft timeout
  expires, the HTTPS-CONNECT filter checks if the QUIC filter
  has received any data from the server. If not, it will start
  the HTTP/2 attempt.

HTTP/3(ngtcp2) improvements.
- setting call_data in all cfilter calls similar to http/2 and vtls filters
  for use in callback where no stream data is available.
- returning CURLE_PARTIAL_FILE for prematurely terminated transfers
- enabling pytest test_05 for h3
- shifting functionality to "connect" UDP sockets from ngtcp2
  implementation into the udp socket cfilter. Because unconnected
  UDP sockets are weird. For example they error when adding to a
  pollset.

HTTP/3(quiche) improvements.
- fixed upload bug in quiche implementation, now passes 251 and pytest
- error codes on stream RESET
- improved debug logs
- handling of DRAIN during connect
- limiting pending event queue

HTTP/2 cfilter improvements.
- use LOG_CF macros for dynamic logging in debug build
- fix CURLcode on RST streams to be CURLE_PARTIAL_FILE
- enable pytest test_05 for h2
- fix upload pytests and improve parallel transfer performance.

GOAWAY handling for ngtcp2/quiche
- during connect, when the remote server refuses to accept new connections
  and closes immediately (so the local conn goes into DRAIN phase), the
  connection is torn down and a another attempt is made after a short grace
  period.
  This is the behaviour observed with nghttpx when we tell it to  shut
  down gracefully. Tested in pytest test_03_02.

TLS improvements
- ALPN selection for SSL/SSL-PROXY filters in one vtls set of functions, replaces
  copy of logic in all tls backends.
- standardized the infof logging of offered ALPNs
- ALPN negotiated: have common function for all backends that sets alpn proprty
  and connection related things based on the negotiated protocol (or lack thereof).

- new tests/tests-httpd/scorecard.py for testing h3/h2 protocol implementation.
  Invoke:
    python3 tests/tests-httpd/scorecard.py --help
  for usage.

Improvements on gathering connect statistics and socket access.
- new CF_CTRL_CONN_REPORT_STATS cfilter control for having cfilters
  report connection statistics. This is triggered when the connection
  has completely connected.
- new void Curl_pgrsTimeWas(..) method to report a timer update with
  a timestamp of when it happend. This allows for updating timers
  "later", e.g. a connect statistic after full connectivity has been
  reached.
- in case of HTTP eyeballing, the previous changes will update
  statistics only from the filter chain that "won" the eyeballing.
- new cfilter query CF_QUERY_SOCKET for retrieving the socket used
  by a filter chain.
  Added methods Curl_conn_cf_get_socket() and Curl_conn_get_socket()
  for convenient use of this query.
- Change VTLS backend to query their sub-filters for the socket when
  checks during the handshake are made.

HTTP/3 documentation on how https eyeballing works.

TLS improvements
- ALPN selection for SSL/SSL-PROXY filters in one vtls set of functions, replaces
  copy of logic in all tls backends.
- standardized the infof logging of offered ALPNs
- ALPN negotiated: have common function for all backends that sets alpn proprty
  and connection related things based on the negotiated protocol (or lack thereof).

Scorecard with Caddy.
- configure can be run with `--with-test-caddy=path` to specify which caddy to use for testing
- tests/tests-httpd/scorecard.py now measures download speeds with caddy

pytest improvements
- adding Makfile to clean gen dir
- adding nghttpx rundir creation on start
- checking httpd version 2.4.55 for test_05 cases where it is needed. Skipping with message if too old.
- catch exception when checking for caddy existance on system.

Closes #10349
2023-02-02 09:57:34 +01:00
Stefan Eissing
33ac97e1cb
tests-httpd: basic infra to run curl against an apache httpd plus nghttpx for h3
- adding '--with-test-httpd=<path>' to configure non-standard apache2
  install
- python env and base classes for running httpd
- basic tests for connectivity with h1/h2/h3
- adding test cases for truncated responses in http versions.
- adding goaway test for HTTP/3.
- adding "stuttering" tests with parallel downloads in chunks with
  varying delays between chunks.

- adding a curltest module to the httpd server, adding GOAWAY test.
    - mod_curltest now installs 2 handlers
      - 'echo': writing as response body what came as request body
      - 'tweak': with query parameters to tweak response behaviour
- marked known fails as skip for now

Closes #10175
2023-01-09 17:40:04 +01:00
Daniel Stenberg
2bc1d775f5
copyright: update all copyright lines and remove year ranges
- they are mostly pointless in all major jurisdictions
- many big corporations and projects already don't use them
- saves us from pointless churn
- git keeps history for us
- the year range is kept in COPYING

checksrc is updated to allow non-year using copyright statements

Closes #10205
2023-01-03 09:19:21 +01:00
Daniel Stenberg
5ab72b9a57
HTTP3: mention what needs to be in place to remove EXPERIMENTAL label
Closes #10168
2022-12-28 09:20:12 +01:00
Daniel Stenberg
901392cbb7
urlapi: add CURLU_PUNYCODE
Allows curl_url_get() get the punycode version of host names for the
host name and URL parts.

Extend test 1560 to verify.

Closes #10109
2022-12-26 23:29:23 +01:00
Viktor Szakats
a8861b6ccd
Makefile.mk: portable Makefile.m32
Update bare GNU Make `Makefile.m32` to:

- Move objects into a subdirectory.
- Add support for MS-DOS. Tested with DJGPP.
- Add support for Watt-32 (on MS-DOS).
- Add support for AmigaOS.
- Rename `Makefile.m32` to `Makefile.mk`
- Replace `ARCH` with `TRIPLET`.
- Build `tool_hugehelp.c` proper (when tools are available).
- Drop MS-DOS compatibility macro `USE_ZLIB` (replaced by `HAVE_LIBZ`)
- Add support for `ZLIB_LIBS` to override `-lz`.
- Omit object files when building examples.
- Default `CC` to `gcc` once again, for convenience. (Caveat: compiler
  name `cc` cannot be set now.)
- Set `-DCURL_NO_OLDIES` for examples, like autotools does.
- Delete `makefile.dj` files. Notice the configuration details and
  defaults are not retained with the new method.
- Delete `makefile.amiga` files. A successful build needs a few custom
  options. We're also not retaining all build details from the existing
  Amiga make files.
- Rename `Makefile.m32` to `Makefile.mk` to reflect that they are not
  Windows/MinGW32-specific anymore.
- Add support for new `CFG` options: `-map`, `-debug`, `-trackmem`
- Set `-DNDEBUG` by default.
- Allow using `-DOS=...` in all `lib/config-*.h` headers, syncing this
  with `config-win32.h`.
- Look for zlib parts in `ZLIB_PATH/include` and `ZLIB_PATH/lib`
  instead of bare `ZLIB_PATH`.

Note that existing build configurations for MS-DOS and AmigaOS likely
become incompatible with this change.

Example AmigaOS configuration:
```
export CROSSPREFIX=/opt/amiga/bin/m68k-amigaos-
export CC=gcc
export CPPFLAGS='-DHAVE_PROTO_BSDSOCKET_H'
export CFLAGS='-mcrt=clib2'
export LDFLAGS="${CFLAGS}"
export LIBS='-lnet -lm'
make -C lib -f Makefile.mk
make -C src -f Makefile.mk
```

Example MS-DOS configuration:
```
export CROSSPREFIX=/opt/djgpp/bin/i586-pc-msdosdjgpp-
export WATT_PATH=/opt/djgpp/net/watt
export ZLIB_PATH=/opt/djgpp
export OPENSSL_PATH=/opt/djgpp
export OPENSSL_LIBS='-lssl -lcrypt'
export CFG=-zlib-ssl
make -C lib -f Makefile.mk
make -C src -f Makefile.mk
```

Closes #9764
2022-11-22 08:28:41 +00:00
Alexandre Ferrieux
49798cac83
CURLOPT_QUICK_EXIT: don't wait for DNS thread on exit
Fixes #2975
Closes #9147
2022-11-18 00:17:27 +01:00
Daniel Stenberg
d4fed2a13a
docs: explain the noproxy CIDR notation support
Follow-up to 1e9a538e05

Closes #9818
2022-10-28 09:02:22 +02:00
Daniel Stenberg
19bfaca7cc
spellcheck.words: remove 'github' as an accepted word
Prefer the properly cased version: GitHub

Use markdown for links and GitHub in text.

Closes #9810
2022-10-27 15:53:11 +02:00
Viktor Szakats
7313ffebfe
docs: spelling nits
- MingW -> MinGW (Minimalist GNU for Windows)
- f.e. -> e.g.
- some whitespace and punctuation.

Reviewed-by: Daniel Stenberg

Closes #9622
2022-09-29 21:29:04 +00:00
Daniel Stenberg
72c41f7c8b
GHA: spellcheck
This spellchecker checks markdown files. For this reason this job
converts all man pages in the repository to markdown with pandoc before
the check runs.

The perl script 'cleanspell' filters out details from the man page in
the process, to avoid the spellchecker trying to spellcheck things it
can't. Like curl specific symbols and the SYNOPSIS and EXAMPLE sections
of libcurl man pages.

The spell checker does not check words in sections that are within pre,
strong and em tags.

'spellcheck.words' is a custom word list with additional accepted words.

Closes #9523
2022-09-21 15:20:07 +02:00