Joel Brobecker 63c457b911 gmp-utils: protect gdb_mpz exports against out-of-range values ... The gdb_mpz class currently provides a couple of methods which essentially export an mpz_t value into either a buffer, or an integral type. The export is based on using the mpz_export function which we discovered can be a bit treacherous if used without caution. In particular, the initial motivation for this patch was to catch situations where the mpz_t value was so large that it would not fit in the destination area. mpz_export does not know the size of the buffer, and therefore can happily write past the end of our buffer. While designing a solution to the above problem, I also discovered that we also needed to be careful when exporting signed numbers. In particular, numbers which are larger than the maximum value for a given signed type size, but no so large as to fit in the *unsigned* version with the same size, would end up being exported incorrectly. This is related to the fact that mpz_export ignores the sign of the value being exportd, and assumes an unsigned export. Thus, for such large values, the appears as if mpz_export is able to fit our value into our buffer, but in fact, it does not. Also, I noticed that gdb_mpz::write wasn't taking its unsigned_p parameter, which was a hole. For all these reasons, a new low-level private method called "safe_export" has been added to class gdb_mpz, whose goal is to perform all necessary checks and manipulations for a safe and correct export. As a bonus, this method allows us to factorize the handling of negative value exports. The gdb_mpz::as_integer and gdb_mpz::write methods are then simplified to take advantage of this new safe_export method. gdb/ChangeLog: * gmp-utils.h (gdb_mpz::safe_export): New private method. (gdb_mpz::as_integer): Reimplement using gdb_mpz::safe_export. * gmp-utils.c (gdb_mpz::write): Rewrite using gdb_mpz::safe_export. (gdb_mpz::safe_export): New method. * unittests/gmp-utils-selftests .c (gdb_mpz_as_integer): Update function description. (check_as_integer_raises_out_of_range_error): New function. (gdb_mpz_as_integer_out_of_range): New function. (_initialize_gmp_utils_selftests): Register gdb_mpz_as_integer_out_of_range as a selftest.		2020-12-05 23:56:59 -05:00
..
basic_string_view
optional
array-view-selftests.c
child-path-selftests.c
cli-utils-selftests.c
command-def-selftests.c
common-utils-selftests.c
copy_bitwise-selftests.c
enum-flags-selftests.c
environ-selftests.c
filtered_iterator-selftests.c
format_pieces-selftests.c
function-view-selftests.c
gmp-utils-selftests.c
lookup_name_info-selftests.c
main-thread-selftests.c
memory-map-selftests.c
memrange-selftests.c
mkdir-recursive-selftests.c
observable-selftests.c
offset-type-selftests.c
optional-selftests.c
parse-connection-spec-selftests.c
ptid-selftests.c
rsp-low-selftests.c
scoped_fd-selftests.c
scoped_mmap-selftests.c
scoped_restore-selftests.c
search-memory-selftests.c
string_view-selftests.c
style-selftests.c
tracepoint-selftests.c
tui-selftests.c
unpack-selftests.c
utils-selftests.c
vec-utils-selftests.c
xml-utils-selftests.c