Fix integer overflow in malloc when tcache is enabled [BZ #22375]

When the per-thread cache is enabled, __libc_malloc uses request2size (which
does not perform an overflow check) to calculate the chunk size from the
requested allocation size. This leads to an integer overflow causing malloc
to incorrectly return the last successfully allocated block when called with
a very large size argument (close to SIZE_MAX).

This commit uses checked_request2size instead, removing the overflow.

(cherry picked from commit 34697694e8a93b325b18f25f7dcded55d6baeaf6)

ChangeLog

@@ -1,3 +1,10 @@
+2017-11-30  Arjun Shankar  <arjun@redhat.com>
+
+	[BZ #22375]
+	CVE-2017-17426
+	* malloc/malloc.c (__libc_malloc): Use checked_request2size
+	instead of request2size.
+
 2017-11-02  Florian Weimer  <fweimer@redhat.com>
 
 	[BZ #22332]

NEWS

@@ -39,6 +39,11 @@ Security related changes:
   without GLOB_NOESCAPE, could write past the end of a buffer while
   unescaping user names.  Reported by Tim Rühsen.
 
+  CVE-2017-17426: The malloc function, when called with an object size near
+  the value SIZE_MAX, would return a pointer to a buffer which is too small,
+  instead of NULL.  This was a regression introduced with the new malloc
+  thread cache in glibc 2.26.  Reported by Iain Buclaw.
+
 The following bugs are resolved with this release:
 
   [16750] ldd: Never run file directly.
@@ -72,6 +77,7 @@ The following bugs are resolved with this release:
   [22321] sysconf: Fix missing definition of UIO_MAXIOV on Linux
   [22322] libc: [mips64] wrong bits/long-double.h installed
   [22325] glibc: Memory leak in glob with GLOB_TILDE (CVE-2017-15671)
+  [22375] malloc returns pointer from tcache instead of NULL (CVE-2017-17426)
 
 Version 2.26
 

malloc/malloc.c

@@ -3029,7 +3029,8 @@ __libc_malloc (size_t bytes)
     return (*hook)(bytes, RETURN_ADDRESS (0));
 #if USE_TCACHE
   /* int_free also calls request2size, be careful to not pad twice.  */
-  size_t tbytes = request2size (bytes);
+  size_t tbytes;
+  checked_request2size (bytes, tbytes);
   size_t tc_idx = csize2tidx (tbytes);
 
   MAYBE_INIT_TCACHE ();