Commit Graph

31210 Commits

Author SHA1 Message Date
Daniel Stenberg
5a904a6938
RELEASE-NOTES: spell out that IPFS is via gateway 2023-09-28 09:49:43 +02:00
Daniel Stenberg
915949e89c
RELEASE-NOTES: synced 2023-09-28 09:42:52 +02:00
Daniel Stenberg
c50cbac0f0
tool_operate: avoid strlen() -1 on zero length content from file
Follow-up to 65b563a96a

Closes #11959
2023-09-28 09:24:57 +02:00
Daniel Stenberg
93885cf3a8
tool_operate: fix memory mixups
Switch to plain getenv() from curl_getenv() to avoid the allocation and
having to keep track of which free() or curl_free() that need to be
used.

Coverity found issues and a memory leak.

Follow-up to 65b563a96a

Closes #11959
2023-09-28 09:24:57 +02:00
Viktor Szakats
a7f8d04ee0
curl-functions.m4: fixup recent bad edits
Follow-up to 96c29900bc #11940

Closes #11966
2023-09-27 12:58:49 +00:00
Daniel Stenberg
d14089df01
curl-functions.m4: fix include line
This made the getaddrinfo detection fail, but we did not spot it in the
CI because it graciously falled back to using legacy functions instead!

Follow-up to 96c29900bc (#11940)

Closes #11965
2023-09-27 14:38:42 +02:00
Daniel Stenberg
1f92db87e0
inet_ntop: add typecast to silence Coverity
CID 1024653:  Integer handling issues  (SIGN_EXTENSION)

Suspicious implicit sign extension: "src[i]" with type "unsigned char
const" (8 bits, unsigned) is promoted in "src[i] << (1 - i % 2 << 3)" to
type "int" (32 bits, signed), then sign-extended to type "unsigned long"
(64 bits, unsigned).  If "src[i] << (1 - i % 2 << 3)" is greater than
0x7FFFFFFF, the upper bits of the result will all be 1.

111         words[i/2] |= (src[i] << ((1 - (i % 2)) << 3));

The value will not be greater than 0x7FFFFFFF so this still cannot
happen.

Also, switch to ints here instead of longs. The values stored are 16 bit
so at least no need to use 64 bit variables. Also, longs are 32 bit on
some platforms so this logic still needs to work with 32 bits.

Closes #11960
2023-09-27 13:18:22 +02:00
Daniel Stenberg
bb50368ef8
docs: adapt SEE ALSO sections to new requirements
To please manpage-syntax.pl used by test 1173

Closes #11957
2023-09-27 10:40:32 +02:00
Daniel Stenberg
7000a0e067
manpage-syntax.pl: verify SEE ALSO syntax
- Enforce a single reference per .BR line
- Skip the quotes around the section number for example (3)
- Insist on trailing commas on all lines except the last
- Error on comma on the last SEE ALSO entry

- List the entries alpha-sorted, not enforced just recommended

Closes #11957
2023-09-27 10:40:23 +02:00
Daniel Stenberg
01d8473b25
connect: expire the timeout when trying next
... so that it gets called again immediately and can continue trying
addresses to connect to. Otherwise it might unnecessarily wait for a
while there.

Fixes #11920
Reported-by: Loïc Yhuel
Closes #11935
2023-09-27 10:35:36 +02:00
Daniel Stenberg
afb4b15614
http: remove wrong comment for http_should_fail
Reported-by: Christian Schmitz
Ref: #11936
Closes #11941
2023-09-27 09:15:03 +02:00
Dan Fandrich
f0f421b468 tool_setopt: remove unused function tool_setopt_flags
This function is identical to tool_setopt_bitmask except that it treats
the argument as unsigned.

Closes #11943
2023-09-26 17:55:11 -07:00
Viktor Szakats
1411c5eb33
cmake: add feature checks for memrchr and getifaddrs
- `HAVE_MEMRCHR` for `memrchr`.
- `HAVE_GETIFADDRS` for `getifaddrs`.
  This was present in `lib/curl_config.h.cmake` but missed the detection
  logic.

To match existing autotools feature checks.

Closes #11954
2023-09-26 22:10:28 +00:00
Viktor Szakats
3795fcde99
cmake: move global headers to specific checks
Before this patch we added standard headers unconditionally to the
global list of headers used for feature checks. This is unnecessary
and also doesn't help CMake 'Generate' performance. This patch moves
these headers to each feature check where they are actually needed.
Stop using `stddef.h`, as it seems unnecessary.

I've used autotools' `m4/curl-functions.m4` to figure out these
dependencies.

Also delete checking for the C89 standard header `time.h`, that I
missed in the earlier commit.

Ref: 96c29900bc #11940

Closes #11951
2023-09-26 22:09:47 +00:00
Viktor Szakats
9567c7ef12
src/mkhelp: make generated code pass checksrc
Closes #11955
2023-09-26 22:09:21 +00:00
Viktor Szakats
7370a7c760
tests: show which curl tool runtests.pl is using
To help debugging when there is issue finding or running it.

Closes #11953
2023-09-26 22:09:11 +00:00
Viktor Szakats
64896fefd0
CI/azure: make MAKEFLAGS global to parallelize all jobs
https://dev.azure.com/daniel0244/curl/_build/results?buildId=17528 (before)
https://dev.azure.com/daniel0244/curl/_build/results?buildId=17545 (after, with -j3)

Closes #11952
2023-09-26 22:08:39 +00:00
Viktor Szakats
cf8f6c04df
CI/azure: migrate old mingw MSYS1 jobs to MSYS2
Also delete an accidental variable reference.

Follow-up to 38029101e2

Closes #11945
2023-09-26 22:08:24 +00:00
Daniel Stenberg
8547ae56fa
docs: add see also curl_multi_get_handles to some man pages
Assisted-by: Jay Satiro

Closes #11942
2023-09-26 23:30:50 +02:00
Viktor Szakats
8e74c0729d
cmake: assume _fseeki64 and no fseeko on Windows
`_fseeki64` is present in mingw-w64 1.0 (2011-09-26) headers, and
at least Watcom C 1.9 (2010) headers and MSVS 2008 [1].

`fseeko` is not present in any of these.

(mingw-w64 1.0 also offers `fseeko64`.)

[1] https://github.com/curl/curl/pull/11944#issuecomment-1734995004

Follow-up to 9c7165e96a #11918

Closes #11950
2023-09-26 14:26:13 +00:00
Viktor Szakats
96c29900bc
build: delete checks for C89 standard headers
Delete checks and guards for standard C89 headers and assume these are
available: `stdio.h`, `string.h`, `time.h`, `setjmp.h`, `stdlib.h`,
`stddef.h`, `signal.h`.

Some of these we already used unconditionally, some others we only used
for feature checks.

Follow-up to 9c7165e96a #11918 (for `stdio.h` in CMake)

Closes #11940
2023-09-26 14:25:10 +00:00
Stefan Eissing
d15ffe18d9 multiif.h: remove Curl_multi_dump declaration
Follow-up to d850eea2 which removed the Curl_multi_dump definition.

Closes https://github.com/curl/curl/pull/11946
2023-09-26 03:51:13 -04:00
Jay Satiro
aa6c94c5bf config-win32: define HAVE__FSEEKI64
Follow-up to 9c7165e9 which added an fseeko wrapper to the lib that
calls _fseeki64 if it is available.

Closes https://github.com/curl/curl/pull/11944
2023-09-26 03:45:20 -04:00
Jay Satiro
398c59ae63 docs: explain how PINNEDPUBLICKEY is independent of VERIFYPEER
- Explain that peer verification via CURLOPT_PINNEDPUBLICKEY takes place
  even if peer verification via CURLOPT_SSL_VERIFYPEER is turned off.

The behavior is verified by test2048.

Bug: https://github.com/curl/curl/issues/2935#issuecomment-418371872
Reported-by: claudiusaiz@users.noreply.github.com

Bug: https://github.com/curl/curl/discussions/11910
Reported-by: Hakan Sunay Halil

Closes https://github.com/curl/curl/pull/11930
2023-09-26 03:44:35 -04:00
Stefan Eissing
34cdcb9b96 openssl: improve ssl shutdown handling
- If SSL shutdown is not finished then make an additional call to
  SSL_read to gather additional tracing.

- Fix http2 and h2-proxy filters to forward do_close() calls to the next
  filter.

For example h2 and SSL shutdown before and after this change:

Before:

Curl_conn_close -> cf_hc_close -> Curl_conn_cf_discard_chain ->
ssl_cf_destroy

After:

Curl_conn_close -> cf_hc_close -> cf_h2_close -> cf_setup_close ->
ssl_cf_close

Note that currently the tracing does not show output on the connection
closure handle. Refer to discussion in #11878.

Ref: https://github.com/curl/curl/discussions/11878

Closes https://github.com/curl/curl/pull/11858
2023-09-26 03:40:20 -04:00
Loïc Yhuel
579f09343d
multi: fix small timeouts
Since Curl_timediff rounds down to the millisecond, timeouts which
expire in less than 1ms are considered as outdated and removed from the
list. We can use Curl_timediff_us instead, big timeouts could saturate
but this is not an issue.

Closes #11937
2023-09-26 00:58:47 +02:00
Viktor Szakats
06133d3e9b
cmake: fix stderr initialization in unity builds
Before this patch, in certain build configurations the curl tool may
not have displayed anything (debug, macOS), or crashed at startup
(debug, Windows).

Follow-up to 3f8fc25720
Necessary after 2f17a9b654

Closes #11929
2023-09-25 22:08:22 +00:00
Viktor Szakats
f3303b57ed
cmake: fix missing zlib.h when compiling libcurltool
Came up while testing debug/testing build for Windows. I'm not sure why
it didn't come up in earlier tests with similar config.
`tool_hugehelp.c` might indeed require `zlib.h` and without linking
`CURL_LIBS` to the `curltool` target, CMake doesn't seem to add detected
dependency headers to the compiler command.

```
[ 25%] Building C object src/CMakeFiles/curltool.dir/tool_hugehelp.c.obj
cd .../curl/bld-cmake-llvm-x64/src && /usr/local/opt/llvm/bin/clang
  --target=x86_64-w64-mingw32 --sysroot=/usr/local/opt/mingw-w64/toolchain-x86_64
  -DCURLDEBUG -DCURL_STATICLIB -DHAVE_CONFIG_H -DUNICODE -DUNITTESTS -D_UNICODE
  -I.../curl/include -I.../curl/lib -I.../curl/bld-cmake-llvm-x64/lib
  -I.../curl/bld-cmake-llvm-x64/include -I.../curl/src -Wno-unused-command-line-argument
  -D_UCRT -DDEBUGBUILD -DHAS_ALPN -DUSE_MANUAL=1  -fuse-ld=lld -Wl,-s -static-libgcc
  -lucrt [...] -O3 -DNDEBUG -municode -MD
  -MT src/CMakeFiles/curltool.dir/tool_hugehelp.c.obj
  -MF CMakeFiles/curltool.dir/tool_hugehelp.c.obj.d
  -o CMakeFiles/curltool.dir/tool_hugehelp.c.obj -c .../curl/bld-cmake-llvm-x64/src/tool_hugehelp.c
.../curl/bld-cmake-llvm-x64/src/tool_hugehelp.c:6:10: fatal error: 'zlib.h' file not found
    6 | #include <zlib.h>
      |          ^~~~~~~~
```

Follow-up to 39e7c22bb4

Closes #11927
2023-09-25 22:06:51 +00:00
Viktor Szakats
4db88d90c8
cmake: fix duplicate symbols when linking tests
The linker resolves this automatically in non-unity builds. In unity
builds the linker cannot drop a single object with the duplicates,
resulting in these errors. The root issue is that we started including
certain objects both via both libcurlu and libcurltool libs.

Regression from 39e7c22bb4

Windows errors:
```
[  3%] Linking C executable unit1303.exe
[  3%] Building C object tests/server/CMakeFiles/rtspd.dir/__/__/lib/curl_multibyte.c.obj
../../lib/libcurlu-d.a(unity_0.c.obj): In function `curlx_convert_UTF8_to_wchar':
C:/projects/curl/lib/curl_multibyte.c:44: multiple definition of `curlx_convert_UTF8_to_wchar'
../../src/libcurltool-d.a(unity_0.c.obj):C:/projects/curl/lib/curl_multibyte.c:44: first defined here
../../lib/libcurlu-d.a(unity_0.c.obj): In function `curlx_convert_wchar_to_UTF8':
C:/projects/curl/lib/curl_multibyte.c:66: multiple definition of `curlx_convert_wchar_to_UTF8'
../../src/libcurltool-d.a(unity_0.c.obj):C:/projects/curl/lib/curl_multibyte.c:66: first defined here
../../lib/libcurlu-d.a(unity_0.c.obj): In function `curlx_win32_open':
C:/projects/curl/lib/curl_multibyte.c:92: multiple definition of `curlx_win32_open'
../../src/libcurltool-d.a(unity_0.c.obj):C:/projects/curl/lib/curl_multibyte.c:92: first defined here
../../lib/libcurlu-d.a(unity_0.c.obj): In function `curlx_win32_fopen':
C:/projects/curl/lib/curl_multibyte.c:120: multiple definition of `curlx_win32_fopen'
../../src/libcurltool-d.a(unity_0.c.obj):C:/projects/curl/lib/curl_multibyte.c:120: first defined here
../../lib/libcurlu-d.a(unity_0.c.obj): In function `curlx_win32_stat':
[...]
```
Ref: https://ci.appveyor.com/project/curlorg/curl/builds/48110107/job/nvlhpt9aa4ehny5q#L247

macOS errors:
```
[ 56%] Linking C executable unit1302
duplicate symbol '_curlx_sotouz' in:
    ../../lib/libcurlu.a(unity_0_c.c.o)
    ../../src/libcurltool.a(unity_0_c.c.o)
duplicate symbol '_curlx_sitouz' in:
    ../../lib/libcurlu.a(unity_0_c.c.o)
    ../../src/libcurltool.a(unity_0_c.c.o)
duplicate symbol '_curlx_uztosz' in:
    ../../lib/libcurlu.a(unity_0_c.c.o)
    ../../src/libcurltool.a(unity_0_c.c.o)
[...]
```
with config:
```
  -DCMAKE_UNITY_BUILD=ON \
  -DENABLE_DEBUG=ON -DBUILD_TESTING=ON -DCMAKE_C_FLAGS=-DDEBUGBUILD \
  -DBUILD_SHARED_LIBS=ON \
  -DBUILD_STATIC_LIBS=OFF
```

Closes #11926
2023-09-25 22:05:38 +00:00
Viktor Szakats
a8ebde99f7
cmake: lib CURL_STATICLIB fixes (Windows)
- always define `CURL_STATICLIB` when building libcurl for Windows.

  This disables `__declspec(dllexport)` for exported libcurl symbols.
  In normal mode (hide symbols) these exported symbols are specified
  via `libcurl.def`. When not hiding symbols, all symbols are exported
  by default.

  Regression from 1199308dbc

  Fixes #11844

- fix to omit `libcurl.def` when not hiding private symbols.

  Regression from 2ebc74c36a

- fix `ENABLED_DEBUG=ON` + shared curl tool Windows builds by also
  omitting `libcurl.def` in this case, and exporting all symbols
  instead. This ensures that a shared curl tool can access all debug
  functions which are not normally exported from libcurl DLL.

- delete `INTERFACE_COMPILE_DEFINITIONS "CURL_STATICLIB"` for "objects"
  target.

  Follow-up to 2ebc74c36a

- delete duplicate `BUILDING_LIBCURL` definitions.

- fix `HIDES_CURL_PRIVATE_SYMBOLS` to not overwrite earlier build settings.

  Follow-up to 1199308dbc

Closes #11914
2023-09-25 22:05:29 +00:00
Daniel Stenberg
826a7c4fb1
RELEASE-NOTES: synced 2023-09-25 23:35:49 +02:00
Dan Fandrich
3ed0e34326 tests: fix log directory path in IPFS tests
Hard-coding the log directory name fails with parallel tests.

Follow-up to 65b563a96

Ref: #8805
2023-09-25 12:39:18 -07:00
Daniel Stenberg
9ffd411735
curl_multi_get_handles: get easy handles from a multi handle
Closes #11750
2023-09-25 20:16:58 +02:00
Stefan Eissing
bb4032a152
http: h1/h2 proxy unification
- use shared code for setting up the CONNECT request
  when tunneling, used in HTTP/1.x and HTTP/2 proxying
- eliminate use of Curl_buffer_send() and other manipulations
  of `data->req` or `data->state.ulbuf`

Closes #11808
2023-09-25 20:12:18 +02:00
Natanael Copa
9c7165e96a
lib: use wrapper for curl_mime_data fseek callback
fseek uses long offset which does not match with curl_off_t. This leads
to undefined behavior when calling the callback and caused failure on
arm 32 bit.

Use a wrapper to solve this and use fseeko which uses off_t instead of
long.

Thanks to the nice people at Libera IRC #musl for helping finding this
out.

Fixes #11882
Fixes #11900
Closes #11918
2023-09-25 20:03:09 +02:00
Natanael Copa
b226bd679a
configure: sort AC_CHECK_FUNCS
No functional changes.
2023-09-25 20:03:09 +02:00
Daniel Stenberg
358f7e7577
warnless: remove unused functions
Previously put there for use with the intel compiler

Closes #11932
2023-09-25 17:06:33 +02:00
Daniel Stenberg
367be426d9
GHA/linux: run singleuse to detect single-use global functions
Use --unit for configure --enable-debug builds

Closes #11932
2023-09-25 17:06:09 +02:00
Daniel Stenberg
d27576b2ad
singleuse: add scan for use in other source codes
This should reduce false-positive to almost zero. Checks for presence in
unit tests if --unit is specified, which is intended for debug builds
where unit testing is enabled.

Closes #11932
2023-09-25 17:05:49 +02:00
Daniel Stenberg
d850eea2d0
multi: remove Curl_multi_dump
A debug-only function that is basically never used. Removed to ease the
use of the singleuse script to detect non-static functions not used
outside the file where it is defined.

Closes #11931
2023-09-25 12:35:40 +02:00
Viktor Szakats
72f0607488
tests: fix compiler warnings
Seen with llvm 17 on Windows x64.

```
.../curl/tests/server/rtspd.c:136:13: warning: no previous extern declaration for non-static variable 'logdir' [-Wmissing-variable-declarations]
  136 | const char *logdir = "log";
      |             ^
.../curl/tests/server/rtspd.c:136:7: note: declare 'static' if the variable is not intended to be used outside of this translation unit
  136 | const char *logdir = "log";
      |       ^
.../curl/tests/server/rtspd.c:137:6: warning: no previous extern declaration for non-static variable 'loglockfile' [-Wmissing-variable-declarations]
  137 | char loglockfile[256];
      |      ^
.../curl/tests/server/rtspd.c:137:1: note: declare 'static' if the variable is not intended to be used outside of this translation unit
  137 | char loglockfile[256];
      | ^
.../curl/tests/server/fake_ntlm.c:43:13: warning: no previous extern declaration for non-static variable 'logdir' [-Wmissing-variable-declarations]
   43 | const char *logdir = "log";
      |             ^
.../curl/tests/server/fake_ntlm.c:43:7: note: declare 'static' if the variable is not intended to be used outside of this translation unit
   43 | const char *logdir = "log";
      |       ^
.../curl/src/tool_doswin.c:350:8: warning: possible misuse of comma operator here [-Wcomma]
  350 |     ++d, ++s;
      |        ^
.../curl/src/tool_doswin.c:350:5: note: cast expression to void to silence warning
  350 |     ++d, ++s;
      |     ^~~
      |     (void)( )
```

```
.../curl/tests/libtest/lib540.c:146:27: warning: result of comparison 'long' > 2147483647 is always false [-Wtautological-type-limit-compare]
  146 |         int itimeout = (L > (long)INT_MAX) ? INT_MAX : (int)L;
      |                         ~ ^ ~~~~~~~~~~~~~
1 warning generated.

.../curl/tests/libtest/libntlmconnect.c:195:31: warning: result of comparison 'long' > 2147483647 is always false [-Wtautological-type-limit-compare]
  195 |       int itimeout = (timeout > (long)INT_MAX) ? INT_MAX : (int)timeout;
      |                       ~~~~~~~ ^ ~~~~~~~~~~~~~
1 warning generated.

.../curl/tests/libtest/lib591.c:117:31: warning: result of comparison 'long' > 2147483647 is always false [-Wtautological-type-limit-compare]
  117 |       int itimeout = (timeout > (long)INT_MAX) ? INT_MAX : (int)timeout;
      |                       ~~~~~~~ ^ ~~~~~~~~~~~~~
1 warning generated.
.../curl/tests/libtest/lib597.c:99:31: warning: result of comparison 'long' > 2147483647 is always false [-Wtautological-type-limit-compare]
   99 |       int itimeout = (timeout > (long)INT_MAX) ? INT_MAX : (int)timeout;
      |                       ~~~~~~~ ^ ~~~~~~~~~~~~~
1 warning generated.
```

Seen on macOS Intel:
```
.../curl/tests/server/sws.c:440:64: warning: field precision should have type 'int', but argument has type 'size_t' (aka 'unsigned long') [-Wformat]
          msnprintf(logbuf, sizeof(logbuf), "Got request: %s %.*s HTTP/%d.%d",
                                                             ~~^~
1 warning generated.
```

Closes #11925
2023-09-24 21:52:54 +00:00
Jay Satiro
ab18c04218 url: fix netrc info message
- Fix netrc info message to use the generic ".netrc" filename if the
  user did not specify a netrc location.

- Update --netrc doc to add that recent versions of curl on Windows
  prefer .netrc over _netrc.

Before:
* Couldn't find host google.com in the (nil) file; using defaults

After:
* Couldn't find host google.com in the .netrc file; using defaults

Closes https://github.com/curl/curl/pull/11904
2023-09-24 03:37:13 -04:00
Dan Fandrich
739a9e8e12 wolfssh: do cleanup in Curl_ssh_cleanup
Closes: #11921
2023-09-23 23:15:11 -07:00
Daniel Stenberg
bb83f86a74
tool_listhelp: regenerated
Polished the --ipfs-gateway description

Fixed the --trace-config description

The script also fixed some other small mistakes

Closes #11923
2023-09-24 00:01:47 +02:00
Viktor Szakats
6a85659e7d
Makefile.mk: always set CURL_STATICLIB for lib (Windows)
Also fix to export all symbols in Windows debug builds, making
`-debug-dyn` builds work with `-DCURL_STATICLIB` set.

Ref: https://github.com/curl/curl/pull/11914 (same for CMake)

Closes #11924
2023-09-23 20:42:47 +00:00
Daniel Stenberg
aa9a6a1770
quic: set ciphers/curves the same way regular TLS does
for OpenSSL/BoringSSL

Fixes #11796
Reported-by: Karthikdasari0423 on github
Assisted-by: Jay Satiro
Closes #11836
2023-09-23 11:36:05 +02:00
Daniel Stenberg
1bf2797ba2
test457: verify --max-filesize with chunked encoding 2023-09-23 11:20:25 +02:00
Daniel Stenberg
914e49b9b7
lib: let the max filesize option stop too big transfers too
Previously it would only stop them from getting started if the size is
known to be too big then.

Update the libcurl and curl docs accordingly.

Fixes #11810
Reported-by: Elliot Killick
Assisted-by: Jay Satiro
Closes #11820
2023-09-23 11:20:20 +02:00
Viktor Szakats
38029101e2
mingw: delete support for legacy mingw.org toolchain
Drop support for "old" / "legacy" / "classic" / "v1" / "mingw32" MinGW:
  https://en.wikipedia.org/wiki/MinGW, https://osdn.net/projects/mingw/
Its homepage used to be http://mingw.org/ [no HTTPS], and broken now.
It supported the x86 CPU only and used a old Windows API header and
implib set, often causing issues. It also misses most modern Windows
features, offering old versions of both binutils and gcc (no llvm/clang
support). It was last updated 2 years ago.

curl now relies on toolchains based on the mingw-w64 project:
https://www.mingw-w64.org/  https://sourceforge.net/projects/mingw-w64/
https://www.msys2.org/  https://github.com/msys2/msys2
https://github.com/mstorsjo/llvm-mingw
(Also available via Linux and macOS package managers.)

Closes #11625
2023-09-23 09:12:57 +00:00
Mark Gaiser
65b563a96a
curl: add support for the IPFS protocols:
- ipfs://<cid>
- ipns://<cid>

This allows you tu use ipfs in curl like:
curl ipfs://<cid>
and
curl ipns://<cid>

For more information consult the readme at:
https://curl.se/docs/ipfs.html

Closes #8805
2023-09-23 11:02:10 +02:00