Commit Graph

6379 Commits

Author SHA1 Message Date
Yedaya Katsman
4ff7f5163f
variable.md: make example use expand
I used double quotes since it seemed required for powershell, so this
example works in both (ba)sh and powershell as well as cmd.exe.

Closes #14118
2024-07-08 23:15:56 +02:00
Andy Reitz
400717e4c0
GIT-INFO.md: remove version requirements
Keep them in docs/INTERNALS.md

Bump lowest perl to 5.8

Closes #14112
2024-07-08 23:06:05 +02:00
Viktor Szakats
5fc61a37c1
examples: suppress deprecation warnings locally
Simplify making clean builds by silencing deprecation warnings inside
the example code where these may occur.

Drop related build tweaks/comments from GHA jobs.

Example warning:
```
curl/docs/examples/postit2-formadd.c:65:16: error: 'CURLFORM_COPYNAME' is deprecated: since 7.56.0. Use curl_mime_name() [-Werror=deprecated-declarations]
   65 |                CURLFORM_COPYNAME, "sendfile",
      |                ^~~~~~~~~~~~~~~~~
```
Ref: https://github.com/curl/curl/actions/runs/9841099503/job/27166970904#step:10:829

Closes #14123
2024-07-08 17:22:21 +02:00
Max Mehl
9104bad820
reuse: switch to REUSE 3.2 and REUSE.toml
- remove scripts/copyright.pl

Closes #14107
2024-07-05 20:47:48 +02:00
Yedaya Katsman
0f12ee8b39
curl: move more options to deprecated category
--no-npn, --sslv2, --sslv3

Closes #14109
2024-07-05 20:45:00 +02:00
Daniel Stenberg
2abfc759b9
cmdline-opts: category cleanup
Option cleanups:

 --get is not upload
 --form* are post
 - added several options into ldap, smtp, imap and pop3
 - shortened the category descriptions in the list

category curl fixes:

 --create-dirs removed from 'curl'
 --ftp-create-dirs removed from 'curl'
 --netrc moved to 'auth' from 'curl'
 --netrc-file moved to 'auth' from 'curl'
 --netrc-optional moved to 'auth' from 'curl'
 --no-buffer moved to 'output' from 'curl'
 --no-clobber removed from 'curl'
 --output removed from 'curl'
 --output-dir removed from 'curl'
 --remove-on-error removed from 'curl'

Add a "global" category:

- Made all "global" options set this category

Add a "deprecated" category:

- Moved the deprecated options to it (maybe they should not be in any
 category long term)

Add a 'timeout' category

- Put a number of appropriate options in it

Add an 'ldap' category

- Put the LDAP related option in there

Remove categories "ECH" and "ipfs"

- They should not be categories. Had only one single option each.

Remove category "misc"

- It should not be a category as it is impossible to know when to browse
  it.

--use-ascii moved to ftp and output
--xattr moved to output
--service-name moved to auth

Managen fixes:

- errors if an option is given a category name that is not already setup
  for in code

- verifies that options set `scope: global` also is put in category
  `global´

Closes #14101
2024-07-05 11:05:50 +02:00
Stefan Eissing
18c61aa036
GHA: configure OpenSSL's libdir as 'lib' only
Also mention in HTTP3.md

OpenSSL has a bug that messes the config `--libdir=path` to become the
wrong path in its pkgconfig files. If we just pass `--libdir=lib` it
should avoid this.

Ref: #14099
See also: https://github.com/openssl/openssl/issues/23569

Closes #14102
2024-07-05 10:16:08 +02:00
Yedaya Katsman
62bfcb0c44
help: add flags to output and ssh categories
- Add --output, --remove-on-error, --output-dir and --created-dirs to
  the output help category

- Add --hostpubmd5, --hostpubsha256, --insecure (-k), and --pubkey to
  the ssh help category

Closes #14076
2024-07-04 09:58:23 +02:00
Stefan Eissing
82b32b344e
TODO: remove item about 'SSL_peak'
GnuTLS todo item about using an equivalent of `SSL_peak()`, which
nicely escaped the word checks, is no longer relevant.

We do not use `SSL_peek()` anymore since connection filters were
introduced.

Closes #14091
2024-07-04 09:56:32 +02:00
Daniel Stenberg
38bdef0220
FEATURES.md: refresh
- added lots of missing stuff
- rearranged a little
- remove all footnotes

Closes #14086
2024-07-03 08:35:56 +02:00
Daniel Stenberg
bf9b22a6f7
curl_easy_perform.md: call it network transfer, not file transfer 2024-07-02 22:55:04 +02:00
Daniel Stenberg
75763a3e88
cmdline-opts: shorten six help texts
o --location-trusted
 o --next
 o --parallel-immmediate
 o --pinnedpubkey
 o --proxy-pass
 o --proxy-ssl-allow-beast

Closes #14075
2024-07-02 07:51:57 +02:00
Viktor Szakats
9484beaa49
cmdline-docs: fix --proxy-ca-native example + tidy-ups
Also:
- fix an indentation.
- fix capitalized option in comment.

Closes #14078
2024-07-02 00:09:13 +02:00
Daniel Stenberg
c074ba64a8
code: language cleanup in comments
Based on the standards and guidelines we use for our documentation.

 - expand contractions (they're => they are etc)
 - host name = > hostname
 - file name => filename
 - user name = username
 - man page => manpage
 - run-time => runtime
 - set-up => setup
 - back-end => backend
 - a HTTP => an HTTP
 - Two spaces after a period => one space after period

Closes #14073
2024-07-01 22:58:55 +02:00
Yedaya Katsman
9b683577e1
docs: add RELEASE-TOOLS.md.dist to .gitignore
Closes #14079
2024-07-01 22:49:55 +02:00
Daniel Stenberg
babaf1cbc3
curl_url_set: elaborate on scheme guessing
Explain a little more and refer to the CURLU_NO_GUESS_SCHEME flag
for getting scheme or URL.

Closes #14071
2024-07-01 16:45:58 +02:00
Daniel Stenberg
816ac2a866
docs: misc language polish
- CURLINFO_FILETIME*: improve language
- add '32bit' and '64bit' as bad words, use 32-bit and 64-bit
- mksymbolsmanpage.pl: avoid "will"

Closes #14070
2024-07-01 16:45:17 +02:00
Daniel Stenberg
ecd654e12e
curl_easy_escape: elaborate a little on encoding a URL
Closes #14069
2024-07-01 10:45:59 +02:00
Daniel Stenberg
582429ed47
curl_str[n]equal.md: tidy up text to make them stand-alone
Previously this was one single manpage for two functions but as they are
two separate ones since a while back, they should each clearly document
their single specific functions.

Follow-up to eefcc1bda4

Closes #14068
2024-07-01 09:38:52 +02:00
Daniel Stenberg
e02b43ecf3
curl_mprintf.md: add missing comma 2024-06-30 16:11:52 +02:00
Daniel Stenberg
08fa47317a
CURLOPT_TLSAUTH_PASSWORD/USERNAME.md: language fixups
- relies *on* TLS SRP
- *for* the specific TLS backends

Closes #14061
2024-06-30 16:04:15 +02:00
Daniel Stenberg
29789ae312
docs/libcurl: polish the single-line descriptions
- use imperative form
- use lowercase
- no period
- unify some phrases
- fix curl_multi_socket and curl_multi_socket_all to keep their own
  descriptions

Closes #14062
2024-06-30 16:03:03 +02:00
Daniel Stenberg
6e95e3f7af
RELEASE-PROCEDURE.md: update release date 2024-06-28 14:30:04 +02:00
Junho Choi
ad7a20d506
quic: update to quiche 0.22.0
quiche 0.22.0 will set SONAME in libquiche.so (libquiche.so.0) for
linux/BSDs. Install a symlink with SONAME.

Closes #14030
Closes #14046
2024-06-28 09:16:30 +02:00
Daniel Stenberg
2d3a51e367
managen: introduce "Multi: per-URL"
For -O, -o and -T that are used once per specified URL.

Closes #14045
2024-06-28 08:42:47 +02:00
Daniel Stenberg
282b9fe8ff
KNOWN_BUGS: three new bugs
These have lingered in the issue tracker for a long time without action.
We don't expect any fixes in the near term either. Move them to the
KNOWN_BUGS document.

Closes #12177
Closes #12171
Closes #13350

Closes #14042
2024-06-27 16:49:22 +02:00
Viktor Szakats
1ccdad64ef
CI: add whitespace checker
Fix issues detected.

Also:

- One of the `.vc` files used LF EOLs, while the other didn't.
  Make that one also use LF EOLs, as this is apparently supported by
  `nmake`.

- Drop `.dsw` and `.btn` types from `.gitattributes`.
  The repository doesn't use them.

- Sync section order with the rest of files in
  `tests/certs/EdelCurlRoot-ca.prm`.

- Indent/align `.prm` and `.pem` files.

- Delete dummy `[something]` section from `.prm` and `.pem` files.

Mental note:
MSVC `.sln` files seem to accept spaces for indentation and also support
LF line-endings. I cannot test this and I don't know what's more
convenient when updating them, so left them as-is, with specific
exclusions.

Closes #14031
2024-06-27 13:33:30 +02:00
Stefan Eissing
8d067e61b0
quic: openssl quic, cmake and doc version update to 3.3.0
Closes #14028
2024-06-26 23:10:50 +02:00
Daniel Stenberg
10cc302090
TODO: specify which response codes that make -f/--fail return error
Suggestion from the user survey 2024

Closes #14020
2024-06-26 08:41:41 +02:00
Stefan Eissing
c9b95c0bb3
lib: graceful connection shutdown
When libcurl discards a connection there are two phases this may go
through: "shutdown" and "closing". If a connection is aborted, the
shutdown phase is skipped and it is closed right away.

The connection filters attached to the connection implement the phases
in their `do_shutdown()` and `do_close()` callbacks. Filters carry now a
`shutdown` flags next to `connected` to keep track of the shutdown
operation.

Filters are shut down from top to bottom. If a filter is not connected,
its shutdown is skipped. Notable filters that *do* something during
shutdown are HTTP/2 and TLS. HTTP/2 sends the GOAWAY frame. TLS sends
its close notify and expects to receive a close notify from the server.

As sends and receives may EAGAIN on the network, a shutdown is often not
successful right away and needs to poll the connection's socket(s). To
facilitate this, such connections are placed on a new shutdown list
inside the connection cache.

Since managing this list requires the cooperation of a multi handle,
only the connection cache belonging to a multi handle is used. If a
connection was in another cache when being discarded, it is removed
there and added to the multi's cache. If no multi handle is available at
that time, the connection is shutdown and closed in a one-time,
best-effort attempt.

When a multi handle is destroyed, all connection still on the shutdown
list are discarded with a final shutdown attempt and close. In curl
debug builds, the environment variable `CURL_GRACEFUL_SHUTDOWN` can be
set to make this graceful with a timeout in milliseconds given by the
variable.

The shutdown list is limited to the max number of connections configured
for a multi cache. Set via CURLMOPT_MAX_TOTAL_CONNECTIONS. When the
limit is reached, the oldest connection on the shutdown list is
discarded.

- In multi_wait() and multi_waitfds(), collect all connection caches
  involved (each transfer might carry its own) into a temporary list.
  Let each connection cache on the list contribute sockets and
  POLLIN/OUT events it's connections are waiting for.

- in multi_perform() collect the connection caches the same way and let
  them peform their maintenance. This will make another non-blocking
  attempt to shutdown all connections on its shutdown list.

- for event based multis (multi->socket_cb set), add the sockets and
  their poll events via the callback. When `multi_socket()` is invoked
  for a socket not known by an active transfer, forward this to the
  multi's cache for processing. On closing a connection, remove its
  socket(s) via the callback.

TLS connection filters MUST NOT send close nofity messages in their
`do_close()` implementation. The reason is that a TLS close notify
signals a success. When a connection is aborted and skips its shutdown
phase, the server needs to see a missing close notify to detect
something has gone wrong.

A graceful shutdown of FTP's data connection is performed implicitly
before regarding the upload/download as complete and continuing on the
control connection. For FTP without TLS, there is just the socket close
happening. But with TLS, the sent/received close notify signals that the
transfer is complete and healthy. Servers like `vsftpd` verify that and
reject uploads without a TLS close notify.

- added test_19_* for shutdown related tests
- test_19_01 and test_19_02 test for TCP RST packets
  which happen without a graceful shutdown and should
  no longer appear otherwise.
- add test_19_03 for handling shutdowns by the server
- add test_19_04 for handling shutdowns by curl
- add test_19_05 for event based shutdowny by server
- add test_30_06/07 and test_31_06/07 for shutdown checks
  on FTP up- and downloads.

Closes #13976
2024-06-26 08:33:17 +02:00
Viktor Szakats
2d5aea9c93
tidy-up: more whitespace
Closes #14014
2024-06-25 14:40:44 +02:00
Daniel Stenberg
1f1975b84c
cmdline-docs: "added in" cleanups
- markup fixes
- remove some mentions of < 7.60.0 changes

Closes #14003
2024-06-24 16:56:52 +02:00
Daniel Stenberg
7628502dff
managen: "added in" fixes
- up the limit: remove all mentions of 7.60 or earlier from manpage
   7.60 is 6 years old now.
 - warn on "broken" added in lines, as they avoid detection
 - fixup added in markup in a few curldown files

Closes #14002
2024-06-24 16:20:03 +02:00
Daniel Stenberg
134d2834fd
docs/cmdline-opts: remove two superfluous "Added in" mentions
The key "added in" phrase for the option itself is added automatically.

Closes #14000
2024-06-24 11:18:53 +02:00
Daniel Stenberg
179c524040
cookie-jar.md: see also --junk-session-cookies
Closes #13996
2024-06-24 11:17:41 +02:00
Daniel Stenberg
fa0b1d9209
TODO: -h option
Support "curl -h --insecure" etc to output the manpage section for the
--insecure command line option in the terminal. Should be possible to
work with either long or short versions of command line options.

Closes #13990
2024-06-23 23:07:13 +02:00
Daniel Stenberg
15e8cebd64
trace-ascii.md: mention "%" for stderr
Closes #13991
2024-06-23 23:06:21 +02:00
Daniel Stenberg
857e334a45
connect-to.md: expand with examples
- add referer from the resolve section to connect-to if user wants
  wildcard for the port number

Closes #13989
2024-06-23 15:52:06 +02:00
Daniel Stenberg
d5777afac6
TODO: connect to multiple IPs in parallel
Closes #13986
2024-06-23 14:31:46 +02:00
Daniel Stenberg
674cbfd613
dump-header.md: mention minus for stdout
Closes #13985
2024-06-23 12:08:58 +02:00
Daniel Stenberg
b84699bf4f
CURLOPT_RESOLVE.md: mention hostname can be wildcard ('*')
Closes #13983
2024-06-23 10:29:22 +02:00
Daniel Stenberg
01159b36c2
KNOWN_BUGS: TFTP tests fail on OpenBSD
Closes #13623
Closes #13975
2024-06-19 12:54:19 +02:00
Daniel Stenberg
b715bb371c
VULN-DISCLOSURE-POLICY: NULL dereferences and crashes
If a malicious server can trigger a NULL dereference in curl or
otherwise cause curl to crash (and nothing worse), chances are big that
we do not consider that a security problem.

Closes #13974
2024-06-19 12:53:35 +02:00
Sergey Markelov
35c0117f47
mbedtls: support CURLOPT_CERTINFO
Closes #13113
2024-06-19 11:33:12 +02:00
brian m. carlson
04739054cd
TODO: TLS channel binding
Closes #13483
2024-06-18 10:30:24 +02:00
Daniel Stenberg
66c4a398d7
CURLOPT_NETRC.md: clarify what it does on Windows
Closes #13956
2024-06-15 17:03:05 +02:00
Daniel Stenberg
4adeb52019
KNOWN_BUGS: "HTTP/2 + TLS spends a lot of time in recv"
Closes #13416
Closes #13955
2024-06-15 10:03:28 +02:00
Yedaya Katsman
6e2d3db99f
examples: add missing binaries to .gitignore
They were showing as changed when built. Add them sorted alphabetically,
while also moving a few more entries to sorted order.

Closes #13952
2024-06-14 13:25:55 +02:00
Yedaya Katsman
9d09f9ec7a
docs: reference non deprecated libcurl options
There are a places where man pages reference deprecated CURLOPT options,
where it doesn't make sense, replace them with the reccomended
replacement option.

also remove reference to the removed mesalink TLS backend

Closes #13951
2024-06-14 13:24:41 +02:00
Daniel Stenberg
b41a1624ce
BINDINGS: update java link to one that exists
The previous java binding seems to have vanished. Link to one that still
exists.

Bug: https://github.com/curl/everything-curl/issues/456
Reported-by: Jiang Wenjian
Closes #13950
2024-06-14 09:47:47 +02:00