Commit Graph

30507 Commits

Author SHA1 Message Date
Stefan Eissing
e024d5665d
lib: add CURLINFO_CONN_ID and CURLINFO_XFER_ID
- add an `id` long to Curl_easy, -1 on init
- once added to a multi (or its own multi), it gets
  a non-negative number assigned by the connection cache
- `id` is unique among all transfers using the same
  cache until reaching LONG_MAX where it will wrap
  around. So, not unique eternally.
- CURLINFO_CONN_ID returns the connection id attached to
  data or, if none present, data->state.lastconnect_id
- variables and type declared in tool for write out

Closes #11185
2023-06-12 23:53:00 +02:00
Daniel Stenberg
fdda99c6ee
CURLOPT_INFILESIZE.3: mention -1 triggers chunked
Ref: #11300
Closes #11304
2023-06-12 23:23:04 +02:00
Philip H
4397e41be0
CI: openssl-3.0.9+quic
Closes #11296
2023-06-12 16:24:52 +02:00
Karthikdasari0423
b8f10b7e7b
HTTP3.md: update openssl version
Closes #11297
2023-06-12 16:23:12 +02:00
Daniel Stenberg
a4a5e438ae
vtls: avoid memory leak if sha256 call fails
... in the pinned public key handling function.

Reported-by: lizhuang0630 on github
Fixes #11306
Closes #11307
2023-06-12 15:57:46 +02:00
Daniel Stenberg
fdfc2bb6be
examples/ipv6: disable on win32
I can't make if_nametoindex() work there

Follow-up to c23dc42f39

Closes #11305
2023-06-12 15:28:35 +02:00
Daniel Stenberg
9ec099a2bc
tool_operate: allow cookie lines up to 8200 bytes
Since this option might set multiple cookies in the same line, it does
not make total sense to cap this at 4096 bytes, which is the limit for a
single cookie name or value.

Closes #11303
2023-06-12 14:15:37 +02:00
Daniel Stenberg
6c25cd4508
test427: verify sending more cookies than fit in a 8190 bytes line
curl will then only populate the header with cookies that fit, dropping
ones that otherwise would have been sent

Ref: https://curl.se/mail/lib-2023-06/0020.html

Closes #11303
2023-06-12 14:15:37 +02:00
Daniel Stenberg
1899899bb7
testutil: allow multiple %-operators on the same line
Closes #11303
2023-06-12 14:15:34 +02:00
Oleg Jukovec
aaba783d33
docs: update CURLOPT_UPLOAD.3
The behavior of CURLOPT_UPLOAD differs from what is described in the
documentation. The option automatically adds the 'Transfer-Encoding:
chunked' header if the upload size is unknown.

Closes #11300
2023-06-12 09:26:41 +02:00
Daniel Stenberg
2738927bb7
RELEASE-NOTES: synced 2023-06-12 09:16:55 +02:00
Daniel Stenberg
818a347325
CURLOPT_AWS_SIGV4.3: remove unused variable from example
Closes #11302
2023-06-12 09:13:55 +02:00
Daniel Stenberg
555bacd6d5
examples/https.c: use CURLOPT_CA_CACHE_TIMEOUT
for demonstration purposes

Closes #11290
2023-06-11 00:17:41 +02:00
Daniel Stenberg
c23dc42f39
example/ipv6: feature CURLOPT_ADDRESS_SCOPE in use
Closes #11282
2023-06-11 00:17:03 +02:00
Karthikdasari0423
8460ef4fa0 docs: Update HTTP3.md for newer ngtcp2 and nghttp3
Follow-up to fb9b9b58

Ref: #11184
Closes #11295
2023-06-10 11:06:24 -07:00
Dan Fandrich
fb9b9b5882 docs: update the supported ngtcp2 and nghttp3 versions
Follow-up to cae9d10b

Ref: #11184
Closes #11294
2023-06-10 08:27:48 -07:00
Dan Fandrich
b65086a83a tests: fix error messages & handling around sockets
The wrong error code was checked on Windows on UNIX socket failures,
which could have caused all UNIX sockets to be reported as having
errored and the tests therefore skipped. Also, a useless error message
was displayed on socket errors in many test servers on Windows because
strerror() doesn't work on WinSock error codes; perror() is overridden
there to work on all errors and is used instead.

Ref #11258
Closes #11265
2023-06-09 15:05:19 -07:00
Daniel Stenberg
9e75932358
CURLOPT_SSH_PRIVATE_KEYFILE.3: expand on the file search
Reported-by: atjg on github
Ref: #11287
Closes #11289
2023-06-09 23:26:08 +02:00
Stefan Eissing
3f784980cb
ngtcp2: use ever increasing timestamp in io
- ngtcp2 v0.16.0 asserts that timestamps passed to its function
  will only ever increase.
- Use a context shared between ingress/egress operations that
  uses a shared timestamp, regularly updated during calls.

Closes #11288
2023-06-09 20:50:31 +02:00
Daniel Stenberg
7630055a32
GHA: use nghttp2 1.54.0 for the ngtcp2 jobs 2023-06-09 20:50:31 +02:00
Philip H
32f103b0e9
GHA: ngtcp2: use 0.16.0 and nghttp3 0.12.0 2023-06-09 20:50:31 +02:00
Daniel Stenberg
cae9d10be3
ngtcp2: build with 0.16.0 and nghttp3 0.12.0
- moved to qlog_write
- crypto => encryption
- CRYPTO => ENCRYPTION
- removed "_is_"
- ngtcp2_conn_shutdown_stream_read and
  ngtcp2_conn_shutdown_stream_write got flag arguments
- the nghttp3_callbacks struct got a recv_settings callback

Closes #11184
2023-06-09 20:50:24 +02:00
Daniel Stenberg
ad8a8e048e
example/http2-download: set CURLOPT_BUFFERSIZE
Primarily because no other example sets it, and remove the disabling of
the certificate check because we should not recommend that.

Closes #11284
2023-06-09 16:48:20 +02:00
Daniel Stenberg
578becba30
example/crawler: also set CURLOPT_AUTOREFERER
Could make sense, and it was not used in any example before.

Closes #11283
2023-06-09 16:47:20 +02:00
Wyatt OʼDay
ce421a7a7f
tls13-ciphers.d: include Schannel
Closes #11271
2023-06-09 16:44:24 +02:00
Daniel Stenberg
259ee6defc
curl_pushheader_byname/bynum.3: document in their own man pages
These two functions were added in 7.44.0 when CURLMOPT_PUSHFUNCTION was
introduced but always lived a life in the shadows, embedded in the
CURLMOPT_PUSHFUNCTION man page. Until now.

It makes better sense and gives more visibility to document them in
their own stand-alone man pages.

Closes #11286
2023-06-09 14:25:47 +02:00
Daniel Stenberg
6e26666d6b
curl_mprintf.3: minor fix of the example 2023-06-09 11:29:41 +02:00
Daniel Stenberg
dacd25888f
curl_url_set: enforce the max string length check for all parts
Update the docs and test 1559 accordingly

Closes #11273
2023-06-08 23:40:08 +02:00
Daniel Stenberg
67e9e90f96
examples/ftpuploadresume.c: add use of CURLOPT_ACCEPTTIMEOUT_MS
For show

Closes #11277
2023-06-08 23:37:43 +02:00
Daniel Stenberg
4894ca6813
examples/unixsocket.c: example using CURLOPT_UNIX_SOCKET_PATH
and alternatively CURLOPT_ABSTRACT_UNIX_SOCKET

Closes #11276
2023-06-08 23:36:55 +02:00
Anssi Kolehmainen
c72edfa8db
docs: fix missing parameter names in examples
Closes #11278
2023-06-08 19:57:08 +02:00
Daniel Stenberg
3c9256c8a0
urlapi: have *set(PATH) prepend a slash if one is missing
Previously the code would just do that for the path when extracting the
full URL, which made a subsequent curl_url_get() of the path to
(unexpectedly) still return it without the leading path.

Amend lib1560 to verify this. Clarify the curl_url_set() docs about it.

Bug: https://curl.se/mail/lib-2023-06/0015.html
Closes #11272
Reported-by: Pedro Henrique
2023-06-08 16:08:45 +02:00
Dan Fandrich
7d62f0d9b8 runtests; give each server a unique log lock file
Logs are written by several servers and all of them must be finished
writing before the test results can be determined. This means each
server must have its own lock file rather than sharing a single one,
which is how it was done up to now. Previously, the first server to
complete a test would clear the lock before the other server was done,
which caused flaky tests.

Lock files are now all found in their own directory, so counting locks
equals counting the files in that directory.  The result is that the
proxy logs are now reliably written which actually changes the expected
output for two tests.

Fixes #11231
Closes #11259
2023-06-07 16:02:58 -07:00
Dan Fandrich
7af151ded0 runtests: make test file directories in log/N
Test files in subdirectories were not created after parallel test log
directories were moved down a level due to a now-bad comparison.

Follow-up to 92d7dd39

Ref #11264
Closes #11267
2023-06-07 15:09:35 -07:00
Daniel Stenberg
78886afb50
ws: make the curl_ws_meta() return pointer a const
The returned info is read-only for the user.

Closes #11261
2023-06-07 23:37:21 +02:00
Daniel Stenberg
44296dc6ec
RELEASE-NOTES: synced 2023-06-07 23:35:51 +02:00
Daniel Stenberg
92d7dd3955
runtests: move parallel log dirs from logN to log/N
Having several hundreds of them in there gets annoying.

Closes #11264
2023-06-07 23:32:41 +02:00
Dan Fandrich
c6d97bcea6 test447: move the test file into %LOGDIR 2023-06-07 13:53:08 -07:00
Viktor Szakats
3f8fc25720
cmake: add support for "unity" builds
Aka "jumbo" or "amalgamation" builds. It means to compile all sources
per target as a single C source. This is experimental.

You can enable it by passing `-DCMAKE_UNITY_BUILD=ON` to cmake.
It requires CMake 3.16 or newer.

It makes builds (much) faster, allows for better optimizations and tends
to promote less ambiguous code.

Also add a new AppVeyor CI job and convert an existing one to use
"unity" mode (one MSVC, one MinGW), and enable it for one macOS CI job.

Fix related issues:
- add missing include guard to `easy_lock.h`.
- rename static variables and functions (and a macro) with names reused
  across sources, or shadowed by local variables.
- add an `#undef` after use.
- add a missing `#undef` before use.
- move internal definitions from `ftp.h` to `ftp.c`.
- `curl_memory.h` fixes to make it work when included repeatedly.
- stop building/linking curlx bits twice for a static-mode curl tool.
  These caused doubly defined symbols in unity builds.
- silence missing extern declarations compiler warning for ` _CRT_glob`.
- fix extern declarations for `tool_freq` and `tool_isVistaOrGreater`.
- fix colliding static symbols in debug mode: `debugtime()` and
  `statename`.
- rename `ssl_backend_data` structure to unique names for each
  TLS-backend, along with the `ssl_connect_data` struct member
  referencing them. This required adding casts for each access.
- add workaround for missing `[P]UNICODE_STRING` types in certain Windows
  builds when compiling `lib/ldap.c`. To support "unity" builds, we had
  to enable `SCHANNEL_USE_BLACKLISTS` for Schannel (a Windows
  `schannel.h` option) _globally_. This caused an indirect inclusion of
  Windows `schannel.h` from `ldap.c` via `winldap.h` to have it enabled
  as well. This requires `[P]UNICODE_STRING` types, which is apperantly
  not defined automatically (as seen with both MSVS and mingw-w64).
  This patch includes `<subauth.h>` to fix it.
  Ref: https://github.com/curl/curl/runs/13987772013
  Ref: https://dev.azure.com/daniel0244/curl/_build/results?buildId=15827&view=logs&jobId=2c9f582d-e278-56b6-4354-f38a4d851906&j=2c9f582d-e278-56b6-4354-f38a4d851906&t=90509b00-34fa-5a81-35d7-5ed9569d331c
- tweak unity builds to compile `lib/memdebug.c` separately in memory
  trace builds to avoid PP confusion.
- force-disable unity for test programs.
- do not compile and link libcurl sources to libtests _twice_ when libcurl
  is built in static mode.

KNOWN ISSUES:
- running tests with unity builds may fail in cases.
- some build configurations/env may not compile in unity mode. E.g.:
  https://ci.appveyor.com/project/curlorg/curl/builds/47230972/job/51wfesgnfuauwl8q#L250

Ref: https://github.com/libssh2/libssh2/issues/1034
Ref: https://cmake.org/cmake/help/latest/prop_tgt/UNITY_BUILD.html
Ref: https://en.wikipedia.org/wiki/Unity_build

Closes #11095
2023-06-07 13:06:08 +00:00
Daniel Stenberg
e812473d1e
examples/websocket.c: websocket example using CONNECT_ONLY
Closes #11262
2023-06-07 14:56:02 +02:00
Daniel Stenberg
0e339b9e72
websocket-cb: example doing WebSocket download using callback
Very basic

Closes #11260
2023-06-07 14:52:00 +02:00
Daniel Stenberg
cd18e5c464
test/.gitignore: ignore log* 2023-06-07 14:42:48 +02:00
Dan Fandrich
6e4fedeef7 runtests: document the -j parallel testing option
Reported-by: Daniel Stenberg
Ref: #10818
Closes #11255
2023-06-05 13:39:44 -07:00
Dan Fandrich
f24b4b980d runtests: create multiple test runners when requested
Parallel testing is enabled by using a nonzero value for the -j option
to runtests.pl. Performant values seem to be about 7*num CPU cores, or
1.3*num CPU cores if Valgrind is in use.

Flaky tests due to improper log locking (bug #11231) are exacerbated
while parallel testing, so it is not enabled by default yet.

Fixes #10818
Closes #11246
2023-06-05 11:16:06 -07:00
Dan Fandrich
d454af4737 runtests: handle repeating tests in multiprocess mode
Such as what happens with the --repeat option.  Some functions are
changed to pass the runner ID instead of relying on the non-unique test
number.

Ref: #10818
2023-06-05 11:16:06 -07:00
Dan Fandrich
51c22af890 runtests: buffer logmsg while running singletest()
This allows all messages relating to a single test case to be displayed
together at the end of the test.

Ref: #10818
2023-06-05 11:16:06 -07:00
Dan Fandrich
296baf45a0 runtests: call initserverconfig() in the runner
This must be done so variables pick up the runner's unique $LOGDIR.

Ref: #10818
2023-06-05 11:16:06 -07:00
Dan Fandrich
78d8bc4c63 runtests: use a per-runner random seed
Each runner needs a unique random seed to reduce the chance of port
number collisions. The new scheme uses a consistent per-runner source of
randomness which results in deterministic behaviour, as it did before.

Ref: #10818
2023-06-05 11:16:06 -07:00
Dan Fandrich
4317c5549b runtests: complete main test loop refactor for multiple runners
The main test loop is now able to handle multiple runners, or no
additional runner processes at all. At most one process is still
created, however.

Ref: #10818
2023-06-05 11:16:06 -07:00
Dan Fandrich
acc0a92897 runtests: prepare main test loop for multiple runners
Some variables are expanded to arrays and hashes so that multiple
runners can be used for running tests.

Ref: #10818
2023-06-05 11:16:06 -07:00