make mmap fail on requests larger than PTRDIFF_MAX
Allocations larger than PTRDIFF_MAX can be successfully created on
32-bit with a 3:1 split, or in 32-bit processes running on 64-bit.
Allowing these allocations to succeed is dangerous, as it introduces
overflows for `end - start` and isn't compatible with APIs (mis)using
ssize_t to report either the size or an error. POSIX is guilty of this,
as are many other Android APIs. LLVM even considers the `ptr + size`
case to be undefined, as all pointer arithmetic compiles down to signed
operations and overflow is treated as undefined for standard C pointer
arithmetic (GNU C `void *` arithmetic works differently).
This also prevents dlmalloc from allocating > PTRDIFF_MAX as it doesn't
merge mappings like jemalloc. A similar check will need to be added in
jemalloc's code path for huge allocations.
The musl libc implementation also performs this sanity check.
Change-Id: I5f849543f94a39719f5d27b00cef3079bb5933e9
diff --git a/libc/bionic/mmap.cpp b/libc/bionic/mmap.cpp
index 8f25a89..794f50f 100644
--- a/libc/bionic/mmap.cpp
+++ b/libc/bionic/mmap.cpp
@@ -30,6 +30,7 @@
#include <sys/mman.h>
#include <unistd.h>
+#include "private/bionic_macros.h"
#include "private/ErrnoRestorer.h"
// mmap2(2) is like mmap(2), but the offset is in 4096-byte blocks, not bytes.
@@ -45,6 +46,13 @@
return MAP_FAILED;
}
+ // prevent allocations large enough for `end - start` to overflow
+ size_t rounded = BIONIC_ALIGN(size, PAGE_SIZE);
+ if (rounded < size || size > PTRDIFF_MAX) {
+ errno = ENOMEM;
+ return MAP_FAILED;
+ }
+
bool is_private_anonymous = (flags & (MAP_PRIVATE | MAP_ANONYMOUS)) != 0;
void* result = __mmap2(addr, size, prot, flags, fd, offset >> MMAP2_SHIFT);