Commit Graph

673 Commits

Author SHA1 Message Date
Daniel Stenberg
a6e8ead533
var: add a '64dec' function that can base64 decode a string
Verified in test 455 and 487.

If the provided string cannot be base64-decoded, it will instead use
"[64dec-fail]" (without the quotes).

Documented

Ref: #16288
Closes #16330
2025-02-22 11:38:52 +01:00
Stefan Eissing
54c07fd6c5
curl: add write-out variable 'tls_earlydata'
Add the write-out variable 'tls_earlydata' to inspect the amount of
TLSv1.3 early data that had been sent.

Closes #15956
2025-02-22 10:02:39 +01:00
Jay Satiro
2335cbaa21 ca-native.md: sync with CURLSSLOPT_NATIVE_CA
- Add that the native CA store is used to verify certs in addition to
  the other certificate location settings.

Basically clarify that --ca-native does not override --cacert etc.

Prior to this change that behavior was only documented in
CURLSSLOPT_NATIVE_CA which is what --ca-native maps to.

Ref: https://github.com/curl/curl/pull/16181#issuecomment-2663998865

Closes https://github.com/curl/curl/pull/16373
2025-02-19 03:39:52 -05:00
Daniel Stenberg
2f4dc6525c
hostip: make CURLOPT_RESOLVE support replacing IPv6 addresses
This also applies to --resolve of course.

Applied strparse functions on the function.

Fixes #16357
Reported-by: rmg-x on github
Closes #16358
Assisted-by: Jay Satiro
2025-02-18 08:55:56 +01:00
Jay Satiro
4a74135737 variable.md: clarify 'trim' example
- Use the variable name 'var' instead of 'url' since the latter is also
  a function name and that may confuse the user.

Closes https://github.com/curl/curl/pull/16346
2025-02-16 18:52:48 -05:00
Daniel Stenberg
c561c94ec3
managen: correct the warning for un-escaped '<' and '>'
1. make sure the check is done before the backticks are replaced

2. ignore less-than and greater-than used within backticks

(adjust proxy.md that now showed a two-space warning)

Closes #16315
2025-02-13 11:17:40 +01:00
Jay Satiro
42960ebec0 write-out.md: add 'header' and 'output' to the variable list
Prior to this change %header{} and %output{} were explained in remarks
but not listed in the --write-out variable list.

Closes https://github.com/curl/curl/pull/16299
2025-02-12 17:49:37 -05:00
Daniel Stenberg
81d253365c
verbose.md: mention how carriage-return might occur in headers
Fixes #16285
Reported-by: Andrei Korshikov
Closes #16286
2025-02-11 07:44:36 +01:00
Terence Eden
a042c67df3
docs: use valid example domain names
Replace .site domains and domain.com with valid example domains.

Fixes #16269
Closes #16270
2025-02-09 00:17:05 +01:00
Daniel Stenberg
448e71d1b7
header.md: mention Authorization: and Cookie: special treatment
Fixes #16160
Reported-by: deliciouslytyped on github
Closes #16164
2025-02-06 11:40:56 +01:00
Jay Satiro
3631c24861 docs: better explain multi-part byte range behavior
- Better explain that if the requested range (--range or CURLOPT_RANGE)
  contains multiple ranges then the response contains meta information
  in addition to the requested bytes.

Prior to this change it was noted that a multiple part response was
returned as-is but not what that meant. In particular, meta information
is returned in addition to the requested bytes and that may have been
unexpected.

Reported-by: Ralf A. Timmermann

Fixes https://github.com/curl/curl/issues/16139
Closes https://github.com/curl/curl/pull/16150
2025-02-06 03:09:45 -05:00
Martin Harrigan
f6da27052e
docs/cmdline-opts/location.md: fix typos for location flag
Closes #16110
2025-01-28 14:07:39 +01:00
9cel
c5a736f2c9
docs: document the behavior of -- in the curl command line
Closes #16053
2025-01-20 14:27:05 +01:00
Daniel Stenberg
862244636e
cmdline-opts/version.md: describe multissl, mention SSLS-EXPORT
Closes #15996
2025-01-14 11:08:24 +01:00
Stefan Eissing
515a21f350
vtls: feature ssls-export for SSL session im-/export
Adds the experimental feature `ssls-export` to libcurl and curl for
importing and exporting SSL sessions from/to a file.

* add functions to libcurl API
* add command line option `--ssl-sessions <filename>` to curl
* add documenation
* add support in configure
* add support in cmake
+ add pytest case

Closes #15924
2025-01-08 23:32:07 +01:00
Daniel Stenberg
3eb57d6ba7
docs: use lowercase curl and libcurl
Adjusted badwords to find them.

Plus: make badwords run on all markdown files in the repo and update
markdowns previously unchecked

Closes #15898
2025-01-02 17:15:54 +01:00
Daniel Stenberg
e256d9df6d
docs/cmdline-opts/_ENVIRONMENT.md: minor language fix
Closes #15897
2025-01-02 17:02:26 +01:00
Jakub Jelen
999cc818c5
openssl: add support to use keys and certificates from PKCS#11 provider
In OpenSSL < 3.0, the modularity was provided by mechanism called
"engines". This is supported in curl, but the engines got deprecated
with OpenSSL 3.0 in favor of more versatile providers.

This adds a support for OpenSSL Providers, to use PKCS#11 keys, namely
through the pkcs11 provider. This is done using similar approach as the
engines and this is automatically built in when the OpenSSL 3 and newer
is used.

Signed-off-by: Jakub Jelen <jjelen@redhat.com>

Closes #15587
2025-01-01 14:47:31 +01:00
Daniel Stenberg
c2d37463b9
curl/var: change byte offset syntax for assignments
Follow-up to 40c264db61 after discussions on IRC.

The new style is

  name[0-99]=contents

and

  name[0-99]@filename

A) This does not cause the same problems with old curl versions trying
the new syntax as this way will cause old curls just fail with syntax
error and not risk using the wrong file.

B) Adds the same byte range support for "normal" assigns, which the
previous syntax did not. Thus lets a user get a partial content of a
variable etc.

Added test 790 and 791 to verify non-file assigns with ranges.

Closes #15862
2024-12-30 15:55:54 +01:00
Daniel Stenberg
31c6b50cb2
curl: make --etag-save acknowledge --create-dirs
Add test 693 to verify

Fixes #15730
Suggested-by: Tamir Duberstein
Closes #15732
2024-12-21 14:12:16 +01:00
Stefan Eissing
b06b35154f
getinfo: fix CURLINFO_QUEUE_TIME_T and add 'time_queue' var
Let CURLINFO_QUEUE_TIME_T count only the time a transfer spends queued,
including possible redirect requests.

Add var 'time_queue' for reporting the time in write outs.

Add test for verifying correct reporting.

Closes #15512
2024-12-21 12:04:48 +01:00
Daniel Stenberg
40c264db61
curl: add byte range support to --variable reading from file
Allowing --variable read a portion of provided files, makes curl work on
partial files for any options that accepts strings. Like --data and others.

The byte offset is provided within brackets, with a semicolon separator
like: --variable name@file;[100-200]"

Inspired by #14479
Assisted-by: Manuel Einfalt

Test 784 - 789. Documentation update provided.

Closes #15739
2024-12-21 11:46:27 +01:00
Daniel Stenberg
fc3e1cbc50
hyper: drop support
lib : remove all hyper code
configure: stop detecting hyper
docs: no more mention of hyper
tests: mo more special-handling of hyper builds
CI: no jobs using hyper

Closes #15120
2024-12-21 11:33:05 +01:00
Viktor Szakats
b9895b9d3f
cmake: namespace functions and macros
Prefix (or suffix) curl-defined macro and function names with `curl`.
To avoid collisions with non-curl CMake and to make them recognizable
as curl-specific calls.

Closes #15498
2024-12-16 21:55:00 +01:00
Daniel Stenberg
0af5ce162f
variable.md: mention --expand-variable for variables to variables
To assign a variable using contents from another variable.

Closes #15752
2024-12-16 08:17:22 +01:00
Daniel Stenberg
9294ca8916
variable.md: show function use with examples
Closes #15743
2024-12-14 17:00:23 +01:00
Daniel Stenberg
7347ddc9e2
tool_operate: make --etag-compare always accept a non-existing file
Consider it a blank etag. It allows for more use cases when the file
just might not have been created yet.

Closes #15737
2024-12-13 15:48:27 +01:00
Daniel Stenberg
a300879b63
curl: return error if etag options are used with multiple URLs
And document it.

Add tests 484 and 485

Fixes #15729
Reported-by: Tamir Duberstein
Closes #15731
2024-12-13 10:11:26 +01:00
Daniel Stenberg
0169b80e5c
curl: --continue-at is mutually exclusive with --remove-on-error
Test 482 verifies

Fixes #15645
Reported-by: Harry Sintonen
Closes #15668
2024-12-02 21:22:57 +01:00
Daniel Stenberg
ffbcde0022
curl: --continue-at is mutually exclusive with --no-clobber
Test 481 verifies

Fixes #15645
Reported-by: Harry Sintonen
Closes #15668
2024-12-02 21:22:47 +01:00
Daniel Stenberg
fcb59534e3
curl: --continue-at is mutually exclusive with --range
Allowing both just creates a transfer with behaviors no user can
properly anticipate so better just deny the combo.

Fixes #15646
Reported-by: Harry Sintonen
Closes #15666
2024-12-02 15:36:44 +01:00
Wyatt O'Day
6238888ca7 schannel: remove TLS 1.3 ciphersuite-list support
Drop TLS 1.3 ciphersuite-list support from SChannel because of the
number of bugs in SChannel itself (a closed-source TLS library). TLS 1.3
with SChannel still works, however the ciphersuite negotiation is left
entirely to SChannel.

Bug: https://hackerone.com/reports/2792484
Reported-by: newfunction on hackerone

Fixes https://github.com/curl/curl/issues/15482
Closes https://github.com/curl/curl/pull/15621
2024-11-21 17:09:24 -05:00
Brendon Smith
dada880fc5
docs: document default User-Agent
curl offers a `--user-agent` option for modifying the `User-Agent`
header supplied in its requests.

The man page section for this option explains how to use the
`--user-agent` option, but does not explain which `User-Agent` curl uses
by default.

By default, curl uses curl/VERSION, such as `User-Agent: curl/8.11.0`.

Note that this appears to be different from the libcurl default (no
User-Agent header).

This commit documents the default `User-Agent` in the man page section
for the `--user-agent` option, as well as on the "Art of Scripting"
page.

The `%VERSION` placeholder will be used to insert the current version
as described in the man page generator docs.

Closes #15608
2024-11-18 00:03:10 +01:00
Daniel Stenberg
ca6d3d2e9b
show-headers.md: clarify the headers are saved with the data
Fixes #15605
Reported-by: tkzv on github
Closes #15606
2024-11-17 22:06:21 +01:00
Daniel Stenberg
f4ee7bafda
cmdline/ech.md: formatting cleanups
Reported-by: Samuel Henrique
Fixes #15506
Closes #15517
2024-11-07 23:47:17 +01:00
Daniel Stenberg
1160380e50
docs: clarify FTP over HTTP proxy functionality somewhat
Reported-by: newfunction

Closes #15417
2024-10-26 23:45:32 +02:00
Daniel Stenberg
1056889f91
url.md: clarify
- the specified URL can also get data sent to it
- rephrase the scheme guessing part
- mention target options for each URL for saving data
- mention --remote-name-all
- remove "warning" and make it into normal text

Closes #15396
2024-10-24 14:32:13 +02:00
Dan Fandrich
6847733191 docs: fix a typo in some cipher options 2024-10-18 14:42:34 -07:00
Viktor Szakats
3040971d12
GHA: silence proselint warnings and an error
Fix new issues found by `proselint`.

Also:
- silence this technical warning:
  ```
  :0: DeprecationWarning: /home/runner/.proselintrc was found instead of a JSON file. Rename to /home/runner/.proselintrc.json.
  ```
- fix an input filename.
  `proselints` fails now if an input file is missing.

Reported-by: Jay Satiro
Bug: https://github.com/curl/curl/pull/15291#issuecomment-2410505100
Closes #15293
2024-10-15 16:44:17 +02:00
Dan Fandrich
954177b9da tool_xattr: create the user.creator xattr attribute
This indicates that the file was created by curl which can help a user
determine the origin of a file. Like the other attributes, this is only
enabled with the --xattr option.

Closes #15237
2024-10-11 10:13:37 -07:00
Stefan Eissing
962097b8dd
TLS: TLSv1.3 earlydata support for curl
Based on #14135, implement TLSv1.3 earlydata support for the curl
command line, libcurl and its implementation in GnuTLS.

If a known TLS session announces early data support, and the feature is
enabled *and* it is not a "connect-only" transfer, delay the TLS
handshake until the first request is being sent.

- Add --tls-earldata as new boolean command line option for curl.
- Add CURLSSLOPT_EARLYDATA to libcurl to enable use of the feature.
- Add CURLINFO_EARLYDATA_SENT_T to libcurl, reporting the amount of
  bytes sent and accepted/rejected by the server.

Implementation details:
- store the ALPN protocol selected at the SSL session.
- When reusing the session and enabling earlydata, use exactly
  that ALPN protocol for negoptiation with the server. When the
  sessions ALPN does not match the connections ALPN, earlydata
  will not be enabled.
- Check that the server selected the correct ALPN protocol for
  an earlydata connect. If the server does not confirm or reports
  something different, the connect fails.
- HTTP/2: delay sending the initial SETTINGS frames during connect,
  if not connect-only.

Verification:
- add test_02_32 to verify earlydata GET with nghttpx.
- add test_07_70 to verify earlydata PUT with nghttpx.
- add support in 'hx-download', 'hx-upload' clients for the feature

Assisted-by: ad-chaos on github
Closes #15211
2024-10-11 12:28:22 +02:00
Sebastian Walz
adca93b53c
json.md: cli-option --json is an alias of --data-binary
Closes #15197
2024-10-08 16:15:06 +02:00
Viktor Szakats
948a2b24f9
ech: spelling, whitespace, say --ech default config
Closes #15167
2024-10-07 12:31:21 +02:00
Tatsuhiro Tsujikawa
7048d1d211
docs/cmdline-opts: GnuTLS supports PKCS#11 URI in --cert option
Closes #15075
2024-09-28 18:09:05 +02:00
Daniel Stenberg
b4f7ec71ce
tool_operate: let --create-dirs work for --dump-header as well
Add test 3011 to verify

Proposed-by: Montg0mery on github
Fixes #14941
Closes #14965
2024-09-22 21:27:29 +02:00
Viktor Szakats
22652a5a4c
curl: add options for safe/no CA bundle search (Windows)
Add `CURL_CA_SEARCH_SAFE` build-time option to enable CA bundle search
in the `curl` tool directory. The lookup method was already used to find
`.curlrc` and `_curlrc` (on Windows). On Windows it overrides the unsafe
default `SearchPath()` method.

Enable with:
- cmake: `-DCURL_CA_SEARCH_SAFE=ON`
- autotools: `--enable-ca-search-safe`
- raw: `CPPFLAGS=-DCURL_CA_SEARCH_SAFE`

On Windows, before this patch the whole `PATH` was searched for
a CA bundle. `PATH` may contain unwanted or world-writable locations,
including the current directory. Searching them all is convenient to
pick up any CA bundle, but not secure.

The Muldersoft curl distro implements such CA search via a custom
patch for Windows:
cd652d4792/patch/curl_tool_doswin.diff (L50)

MSYS2/mingw-w64 distro has also been rolling a patch solving this:
https://github.com/msys2/MINGW-packages/blob/master/mingw-w64-curl/0001-Make-cURL-relocatable.patch
https://github.com/msys2/MINGW-packages/blob/master/mingw-w64-curl/pathtools.c

Also add option to fully disable Windows CA search:
- cmake: `-DCURL_DISABLE_CA_SEARCH=ON`
- autotools: `--disable-ca-search`
- raw: `CPPFLAGS=-DCURL_DISABLE_CA_SEARCH`.

Both options are considered EXPERIMENTAL, with possible incompatible
changes or even (partial) removal in the future, depending on feedback.

An alternative, secure option is to embed the CA bundle into the binary.

Safe search can be extended to other platforms if necessary or useful,
by using `_NSGetExecutablePath()` (macOS),
`/proc/self/exe` (Linux/Cygwin), or `argv[0]`.

Closes #14582
2024-09-22 18:17:25 +02:00
Viktor Szakats
496da69aa0
cmake: fix broken dependency chain for cmdline-opts, tidy-ups
- make `curl.1` and `curl.txt` depend on `DPAGES`.
  To trigger a rebuild when an individual manpage is updated.

- tell CMake that the cmdline-opts command also creates `curl.txt`.

- make `tool_hugehelp.c` depend on `curl.txt` (was: `curl.1`), to match
  what it actually uses for input.

- stop using `generate-curl.1` as an indirect way to create `curl.txt`
  in time for `tool_hugehelp.c`. After the fixes above there is a direct
  depedency chain between them.

- move `ASCIIPAGE` and `MANPAGE` variables to top-level, re-use them in
  `src` and prefix them with `CURL_` to avoid clashing with other
  projects.

- drop double quotes from `generate-curl.1` as a hint that it is not
  a filename, but a target name.

- src: tidy up order of dependency lists.

Closes #14883
2024-09-21 00:59:52 +02:00
Viktor Szakats
805bbf7c55
NTLM_WB: delete remains in tests, docs updates
Also update more docs to reflect the removal of NTLM_WB.

Follow-up to 50def7c881 #13249
Closes #14832
2024-09-10 14:13:36 +02:00
Daniel Stenberg
7143833f14
cmdline-opts: language fix for expect100-timeout.md and max-time.md
needs to **be** provided

Also a http3.md spellfix

Follow-up from 22a6a0bc6b

Closes #14737
2024-08-30 11:19:30 +02:00
Daniel Stenberg
22a6a0bc6b
http3.md: mention how the fallback can be h1 or h2
Closes #14736
2024-08-30 10:32:20 +02:00