Commit Graph

288 Commits

Author SHA1 Message Date
Daniel Stenberg
7309b9cbbf
lib: strndup/memdup instead of malloc, memcpy and null-terminate
- bufref: use strndup
 - cookie: use strndup
 - formdata: use strndup
 - ftp: use strndup
 - gtls: use aprintf instead of malloc + strcpy * 2
 - http: use strndup
 - mbedtls: use strndup
 - md4: use memdup
 - ntlm: use memdup
 - ntlm_sspi: use strndup
 - pingpong: use memdup
 - rtsp: use strndup instead of malloc, memcpy and null-terminate
 - sectransp: use strndup
 - socks_gssapi.c: use memdup
 - vtls: use dynbuf instead of malloc, snprintf and memcpy
 - vtls: use strdup instead of malloc + memcpy
 - wolfssh: use strndup

Closes #12453
2023-12-07 08:47:44 +01:00
Daniel Stenberg
2b0994c29a
cookie: lowercase the domain names before PSL checks
Reported-by: Harry Sintonen

Closes #12387
2023-11-23 10:08:56 +01:00
Daniel Stenberg
70e3b534d5
urldata: move cookielist from UserDefined to UrlState
1. Because the value is not strictly set with a setopt option.

2. Because otherwise when duping a handle when all the set.* fields are
   first copied and an error happens (think out of memory mid-function),
   the function would easily free the list *before* it was deep-copied,
   which could lead to a double-free.

Closes #12323
2023-11-15 09:42:30 +01:00
Sam James
bc8509a748
misc: fix -Walloc-size warnings
GCC 14 introduces a new -Walloc-size included in -Wextra which gives:

```
src/tool_operate.c: In function ‘add_per_transfer’:
src/tool_operate.c:213:5: warning: allocation of insufficient size ‘1’ for type ‘struct per_transfer’ with size ‘480’ [-Walloc-size]
  213 |   p = calloc(sizeof(struct per_transfer), 1);
      |     ^
src/var.c: In function ‘addvariable’:
src/var.c:361:5: warning: allocation of insufficient size ‘1’ for type ‘struct var’ with size ‘32’ [-Walloc-size]
  361 |   p = calloc(sizeof(struct var), 1);
      |     ^
```

The calloc prototype is:
```
void *calloc(size_t nmemb, size_t size);
    ```

So, just swap the number of members and size arguments to match the
prototype, as we're initialising 1 struct of size `sizeof(struct
...)`. GCC then sees we're not doing anything wrong.

Closes #12292
2023-11-11 23:35:47 +01:00
Daniel Stenberg
d3b3ba35a5
lib: add and use Curl_strndup()
The Curl_strndup() function is similar to memdup(), but copies 'n' bytes
then adds a terminating null byte ('\0').

Closes #12251
2023-11-02 20:35:20 +01:00
Daniel Stenberg
73c82ed941
cookie: set ->running in cookie_init even if data is NULL
This is a regression introduced in b1b326ec50 (shipped in curl 8.1.0)

Test 3103 verifies.

Fixes #11875
Reported-by: wangp on github
Closes #11876
2023-09-19 08:26:07 +02:00
Daniel Stenberg
f88cc654ec
cookie: reduce variable scope, add const 2023-09-15 22:09:43 +02:00
Daniel Stenberg
8c285a76ee
cookie: do not store the expire or max-age strings
Convert it to an expire time at once and save memory.

Closes #11862
2023-09-15 22:09:39 +02:00
Daniel Stenberg
61275672b4
cookie: remove unnecessary struct fields
Plus: reduce the hash table size from 256 to 63. It seems unlikely to
make much of a speed difference for most use cases but saves 1.5KB of
data per instance.

Closes #11862
2023-09-15 22:09:06 +02:00
Viktor Szakats
3f8fc25720
cmake: add support for "unity" builds
Aka "jumbo" or "amalgamation" builds. It means to compile all sources
per target as a single C source. This is experimental.

You can enable it by passing `-DCMAKE_UNITY_BUILD=ON` to cmake.
It requires CMake 3.16 or newer.

It makes builds (much) faster, allows for better optimizations and tends
to promote less ambiguous code.

Also add a new AppVeyor CI job and convert an existing one to use
"unity" mode (one MSVC, one MinGW), and enable it for one macOS CI job.

Fix related issues:
- add missing include guard to `easy_lock.h`.
- rename static variables and functions (and a macro) with names reused
  across sources, or shadowed by local variables.
- add an `#undef` after use.
- add a missing `#undef` before use.
- move internal definitions from `ftp.h` to `ftp.c`.
- `curl_memory.h` fixes to make it work when included repeatedly.
- stop building/linking curlx bits twice for a static-mode curl tool.
  These caused doubly defined symbols in unity builds.
- silence missing extern declarations compiler warning for ` _CRT_glob`.
- fix extern declarations for `tool_freq` and `tool_isVistaOrGreater`.
- fix colliding static symbols in debug mode: `debugtime()` and
  `statename`.
- rename `ssl_backend_data` structure to unique names for each
  TLS-backend, along with the `ssl_connect_data` struct member
  referencing them. This required adding casts for each access.
- add workaround for missing `[P]UNICODE_STRING` types in certain Windows
  builds when compiling `lib/ldap.c`. To support "unity" builds, we had
  to enable `SCHANNEL_USE_BLACKLISTS` for Schannel (a Windows
  `schannel.h` option) _globally_. This caused an indirect inclusion of
  Windows `schannel.h` from `ldap.c` via `winldap.h` to have it enabled
  as well. This requires `[P]UNICODE_STRING` types, which is apperantly
  not defined automatically (as seen with both MSVS and mingw-w64).
  This patch includes `<subauth.h>` to fix it.
  Ref: https://github.com/curl/curl/runs/13987772013
  Ref: https://dev.azure.com/daniel0244/curl/_build/results?buildId=15827&view=logs&jobId=2c9f582d-e278-56b6-4354-f38a4d851906&j=2c9f582d-e278-56b6-4354-f38a4d851906&t=90509b00-34fa-5a81-35d7-5ed9569d331c
- tweak unity builds to compile `lib/memdebug.c` separately in memory
  trace builds to avoid PP confusion.
- force-disable unity for test programs.
- do not compile and link libcurl sources to libtests _twice_ when libcurl
  is built in static mode.

KNOWN ISSUES:
- running tests with unity builds may fail in cases.
- some build configurations/env may not compile in unity mode. E.g.:
  https://ci.appveyor.com/project/curlorg/curl/builds/47230972/job/51wfesgnfuauwl8q#L250

Ref: https://github.com/libssh2/libssh2/issues/1034
Ref: https://cmake.org/cmake/help/latest/prop_tgt/UNITY_BUILD.html
Ref: https://en.wikipedia.org/wiki/Unity_build

Closes #11095
2023-06-07 13:06:08 +00:00
Daniel Stenberg
127eb0d83a
misc: fix spelling mistakes
Reported-by: musvaage on github
Fixes #11171
Closes #11172
2023-05-23 10:42:09 +02:00
Emanuele Torre
f198d33e8d
checksrc: disallow spaces before labels
Out of 415 labels throughout the code base, 86 of those labels were
not at the start of the line. Which means labels always at the start of
the line is the favoured style overall with 329 instances.

Out of the 86 labels not at the start of the line:
* 75 were indented with the same indentation level of the following line
* 8 were indented with exactly one space
* 2 were indented with one fewer indentation level then the following
  line
* 1 was indented with the indentation level of the following line minus
  three space (probably unintentional)

Co-Authored-By: Viktor Szakats

Closes #11134
2023-05-18 20:45:04 +02:00
Daniel Stenberg
c79356d852
cookie: address PVS nits
- avoid assigning the same value again
- remove superfluous check of co->domain
- reduce variable scope for namep/valuep

Ref: #10929
Closes #10954
2023-04-14 09:23:04 +02:00
Daniel Stenberg
b1b326ec50
cookie: clarify that init with data set to NULL reads no file
... and make Curl_cookie_add() require 'data' being set proper with an
assert.

The function has not worked with a NULL data for quite some time so this
just corrects the code and comment.

This is a different take than the proposed fixed in #10927

Reported-by: Kvarec Lezki
Ref: #10929
Closes #10930
2023-04-11 23:49:50 +02:00
Daniel Stenberg
45d4bf5af3
cookie: don't load cookies again when flushing
Reported-by: Sergio Mijatovic
Fixes #10677
Closes #10685
2023-03-06 15:51:56 +01:00
Daniel Stenberg
ac5ea7b6eb
cookie: parse without sscanf()
Saves us from using 2*4096 bytes buffers on stack, the extra copies and
more.

Closes #10550
2023-02-20 23:49:51 +01:00
Kvarec Lezki
9caa7bc930
cookies: fp is always not NULL
Closes #10383
2023-02-01 07:57:06 +01:00
Daniel Stenberg
2bc1d775f5
copyright: update all copyright lines and remove year ranges
- they are mostly pointless in all major jurisdictions
- many big corporations and projects already don't use them
- saves us from pointless churn
- git keeps history for us
- the year range is kept in COPYING

checksrc is updated to allow non-year using copyright statements

Closes #10205
2023-01-03 09:19:21 +01:00
Daniel Stenberg
af5999a674
urldata: move the cookefilelist to the 'set' struct
The cookiefile entries are set into the handle and should remain set for
the lifetime of the handle so that duplicating it also duplicates the
list. Therefore, the struct field is moved from 'state' to 'set'.

Fixes #10133
Closes #10134
2022-12-22 16:06:37 +01:00
Daniel Stenberg
73d6f41489
cookie: open cookie jar as a binary file
On Windows there is a difference and for text files, ^Z means end of
file which is not desirable.

Ref: #9973
Closes #10017
2022-12-02 10:07:41 +01:00
Daniel Stenberg
cc3d517e8c
cookie: expire cookies at once when max-age is negative
Update test 329 to verify

Reported-by: godmar on github
Fixes #9930
Closes #9932
2022-11-17 13:47:24 +01:00
Rose
f151ec6c10 lib: fix some type mismatches and remove unneeded typecasts
Many of these castings are unneeded if we change the variables to work
better with each other.

Ref: https://github.com/curl/curl/pull/9823

Closes https://github.com/curl/curl/pull/9835
2022-11-08 03:11:01 -05:00
Daniel Stenberg
14061f784c
cookie: compare cookie prefixes case insensitively
Adapted to language in rfc6265bis draft-11.

Closes #9863

Reviewed-by: Daniel Gustafsson
2022-11-08 09:08:09 +01:00
Ayesh Karunaratne
4484270afc
misc: typo and grammar fixes
- Replace `Github` with `GitHub`.
- Replace `windows` with `Windows`
- Replace `advice` with `advise` where a verb is used.
- A few fixes on removing repeated words.
- Replace `a HTTP` with `an HTTP`

Closes #9802
2022-10-27 10:01:30 +02:00
Daniel Gustafsson
8f384577b8 cookies: optimize control character check
When checking for invalid octets the strcspn() call will return the
position of the first found invalid char or the first NULL byte.
This means that we can check the indicated position in the search-
string saving a strlen() call.

Closes: #9736
Reviewed-by: Jay Satiro <raysatiro@yahoo.com>
2022-10-24 11:31:08 +02:00
Daniel Stenberg
bfe9b59be4
cookie: reject cookie names or content with TAB characters
TABs in name and content seem allowed by RFC 6265: "the algorithm strips
leading and trailing whitespace from the cookie name and value (but
maintains internal whitespace)"

Cookies with TABs in the names are rejected by Firefox and Chrome.

TABs in content are stripped out by Firefox, while Chrome discards the
whole cookie.

TABs in cookies also cause issues in saved netscape cookie files.

Reported-by: Trail of Bits

URL: https://curl.se/mail/lib-2022-10/0032.html
URL: https://github.com/httpwg/http-extensions/issues/2262

Closes #9659
2022-10-14 08:22:10 +02:00
Daniel Stenberg
8dfc93e573
cookie: reject cookies with "control bytes"
Rejects 0x01 - 0x1f (except 0x09) plus 0x7f

Reported-by: Axel Chong

Bug: https://curl.se/docs/CVE-2022-35252.html

CVE-2022-35252

Closes #9381
2022-08-29 11:20:37 +02:00
Daniel Stenberg
4c0ac2aa59
cookie: treat a blank domain in Set-Cookie: as non-existing
This matches what RFC 6265 section 5.2.3 says.

Extended test 31 to verify.

Fixes #9164
Reported-by: Gwen Shapira
Closes #9177
2022-07-19 11:52:54 +02:00
Daniel Stenberg
c5f3c47cbb
cookie: use %zu to infof() for size_t values
Detected by Coverity. CID 1507051
Closes #9095
2022-07-04 14:37:48 +02:00
Daniel Stenberg
20f9dd6bae
fopen: add Curl_fopen() for better overwriting of files
Bug: https://curl.se/docs/CVE-2022-32207.html
CVE-2022-32207
Reported-by: Harry Sintonen
Closes #9050
2022-06-26 11:03:57 +02:00
Daniel Stenberg
48d7064a49
cookie: apply limits
- Send no more than 150 cookies per request
- Cap the max length used for a cookie: header to 8K
- Cap the max number of received Set-Cookie: headers to 50

Bug: https://curl.se/docs/CVE-2022-32205.html
CVE-2022-32205
Reported-by: Harry Sintonen
Closes #9048
2022-06-26 11:00:48 +02:00
max.mehl
ad9bc5976d
copyright: make repository REUSE compliant
Add licensing and copyright information for all files in this repository. This
either happens in the file itself as a comment header or in the file
`.reuse/dep5`.

This commit also adds a Github workflow to check pull requests and adapts
copyright.pl to the changes.

Closes #8869
2022-06-13 09:13:00 +02:00
Daniel Gustafsson
739275a1d9 cookies: fix documentation comment
Commit 4073cd83b2 added the noexpire parameter to Curl_cookie_add but
missed updating the documentation comment at the head of the file.
2022-05-30 13:46:25 +02:00
Daniel Stenberg
dcaae6bb5b
cookie: fix false positive "potentially uninitialized local variable"
Reviewed-by: Daniel Gustafsson
Closes #8903
2022-05-24 12:16:08 +02:00
Harry Sintonen
e9a8451a3b
cookie: address secure domain overlay
Bug: https://hackerone.com/reports/1560324
Co-authored-by: Daniel Stenberg
Closes #8840
2022-05-19 14:48:26 +02:00
Daniel Stenberg
7e92d12b4e
cookies: make bad_domain() not consider a trailing dot fine
The check for a dot in the domain must not consider a single trailing
dot to be fine, as then TLD + trailing dot is fine and curl will accept
setting cookies for it.

CVE-2022-27779

Reported-by: Axel Chong
Bug: https://curl.se/docs/CVE-2022-27779.html
Closes #8820
2022-05-09 16:47:28 +02:00
Daniel Stenberg
4ec0549c39
misc: update copyright year ranges 2022-04-25 09:31:09 +02:00
Daniel Gustafsson
a6cdfd24ee cookies: Improve errorhandling for reading cookiefile
The existing programming had some issues with errorhandling for reading
the cookie file. If the file failed to open, we would silently ignore it
and continue as if there was no file (or stdin) passed. In this case, we
would also call fclose() on the NULL FILE pointer, which is undefined
behavior. Fix by ensuring that the FILE pointer is set before calling
fclose on it, and issue a warning in case the file cannot be opened.
Erroring out on nonexisting file would break backwards compatibility of
very old behavior so we can't really go there.

Closes: #8699
Reviewed-by: Daniel Stenberg <daniel@haxx.se>
Reviewed-by: Jay Satiro <raysatiro@yahoo.com>
2022-04-20 14:17:29 +02:00
Daniel Stenberg
21248e052d
checksrc: detect more kinds of NULL comparisons we avoid
Co-authored-by: Jay Satiro
Closes #8180
2021-12-27 23:39:26 +01:00
Daniel Stenberg
e7416cfd2b
infof: remove newline from format strings, always append it
- the data needs to be "line-based" anyway since it's also passed to the
  debug callback/application

- it makes infof() work like failf() and consistency is good

- there's an assert that triggers on newlines in the format string

- Also removes a few instances of "..."

- Removes the code that would append "..." to the end of the data *iff*
  it was truncated in infof()

Closes #7357
2021-07-07 22:54:01 +02:00
Andrei Rybak
278b46751e misc: fix typos in comments which repeat a word
Fix typos in code comments which repeat various words.  In trivial
cases, just delete the repeated word.  Reword the affected sentence in
"lib/url.c" for it to make sense.

Closes #7303
Reviewed-by: Daniel Gustafsson <daniel@yesql.se>
2021-06-28 12:41:56 +02:00
Daniel Gustafsson
1bd4b3f4e2 cookies: track expiration in jar to optimize removals
Removing expired cookies needs to be a fast operation since we want to
be able to perform it often and speculatively. By tracking the timestamp
of the next known expiration we can exit early in case the timestamp is
in the future.

Closes: #7172
Reviewed-by: Daniel Stenberg <daniel@haxx.se>
2021-06-08 09:31:19 +02:00
Daniel Gustafsson
002f976cdc cookies: fix typo and expand comment
Fix a typo in the sorting comment, and while in there elaborate slightly
on why creationtime can be used as a tiebreaker.
2021-06-03 13:02:09 +02:00
Daniel Gustafsson
9750bc97d8 cookies: remove unused header
Commit 1c1d9f1aff removed the last use
for the inet_pton.h headerfile, this removes the inclusion of the
header.

Closes: #7182
Reviewed-by: Daniel Stenberg <daniel@haxx.se>
2021-06-03 13:02:07 +02:00
Daniel Stenberg
1c1d9f1aff
hsts: ignore numberical IP address hosts
Also, use a single function library-wide for detecting if a given hostname is
a numerical IP address.

Reported-by: Harry Sintonen
Fixes #7146
Closes #7149
2021-05-30 19:49:40 +02:00
Daniel Gustafsson
24c71d62ee
cookies: use CURLcode for cookie_output reporting
Writing the cookie file has multiple error conditions, and was using an
int with magic numbers to report the different error (which in turn were
disregarded anyways). This moves reporting to use a CURLcode value.

Lightly-touched-by: Daniel Stenberg

Closes #7037
Closes #6749
2021-05-11 08:49:00 +02:00
Daniel Gustafsson
98888e6070
cookies: make use of string duplication function
strstore() is defined as a strdup which ensures to free the target
pointer before duping the source char * into it. Make use of it in
two more cases where it can simplify the code.
2021-05-11 08:45:17 +02:00
Daniel Gustafsson
54bd65cabd
cookies: refactor comments
Comments in the cookie code were a bit all over the place in terms of
style and wording. This takes a stab at cleaning them up by keeping to
a single style and overall shape. Some comments are moved a little and
some removed alltogether due to being redundant. No functional changes
have been made,
2021-05-11 08:45:17 +02:00
Daniel Stenberg
063d3f3b96
tidy-up: make conditional checks more consistent
... remove '== NULL' and '!= 0'

Closes #6912
2021-04-22 09:10:17 +02:00
Daniel Stenberg
95cbcec8f9
urldata: merge "struct DynamicStatic" into "struct UrlState"
Both were used for the same purposes and there was no logical separation
between them. Combined, this also saves 16 bytes in less holes in my
test build.

Closes #6798
2021-03-26 23:19:20 +01:00