ashmem: Only use memfd if the kernel has ashmem-memfd compat support
There are still old applications that have not migrated to using
libcutils, and allocate shared memory buffers by directly allocating
them from "/dev/ashmem", and then using the ashmem ioctl commands
to operate on the buffer.
This works well on a system where all shared memory buffers are ashmem
buffers. However, in a system where there could be a mix of ashmem and
memfd buffers (e.g. if sys.use_memfd == true), this could be a problem
as ashmem ioctl commands on memfds will fail if the kernel does not
have support for ashmem-memfd compatibility.
Therefore, check for ashmem-memfd compatiblity before deciding if memfd
is usable.
Bug: 111903542
Change-Id: I760e57b31b53cdd4cfbf94cb35a6fb47b089bb1b
Signed-off-by: Isaac J. Manjarres <isaacmanjarres@google.com>
diff --git a/libcutils/ashmem-dev.cpp b/libcutils/ashmem-dev.cpp
index 6cb7986..615ef5e 100644
--- a/libcutils/ashmem-dev.cpp
+++ b/libcutils/ashmem-dev.cpp
@@ -132,6 +132,16 @@
return false;
}
+ /*
+ * Ensure that the kernel supports ashmem ioctl commands on memfds. If not,
+ * fall back to using ashmem.
+ */
+ if (TEMP_FAILURE_RETRY(ioctl(fd, ASHMEM_SET_SIZE, getpagesize())) < 0) {
+ ALOGE("ioctl(ASHMEM_SET_SIZE) failed: %s, no ashmem-memfd compat support.\n",
+ strerror(errno));
+ return false;
+ }
+
if (debug_log) {
ALOGD("memfd: device has memfd support, using it\n");
}