fs_mgr: overlayfs: check if scratch device is ext4 dedupe

Do not attempt to mount scratch r/w if it is ext4 dedupe, this causes
too much noise and troubling but innocuous error reports.

Assumption is we normally try f2fs first on all devices, we only try
ext4 first if we do not have f2fs tools, or if the existing
filesystem is ext4.  That said, we only have to check if it is ext4
dedupe during the first mount attempt, the fallback mount attempt for
ext4 is unlikely to need this checking.

Changes the output report for a retrofit DAP device from:

$ adb remount
Disabling verity for /system
[libfs_mgr]superblock s_max_mnt_count:65535,/dev/block/by-name/system_b
[libfs_mgr]check_fs(): mount(/dev/block/by-name/system_b,/mnt/scratch,ext4)=-1: Invalid argument
[libfs_mgr]Running /system/bin/e2fsck on /dev/block/sda6
[libfs_mgr]__mount(source=/dev/block/by-name/system_b,target=/mnt/scratch,type=ext4)=-1: Invalid argument
[libfs_mgr]Running /system/bin/fsck.f2fs -a /dev/block/sda6
[libfs_mgr]__mount(source=/dev/block/by-name/system_b,target=/mnt/scratch,type=f2fs)=0: Success
Using overlayfs for /system
. . .

To the more pleasant:

$ adb $BL1 remount
Disabling verity for /system
[libfs_mgr]superblock s_max_mnt_count:65535,/dev/block/by-name/system_b
[libfs_mgr]__mount(source=/dev/block/by-name/system_b,target=/mnt/scratch,type=ext4)=0: Success
[libfs_mgr]umount(/mnt/scratch)
[libfs_mgr]Running /system/bin/fsck.f2fs -a /dev/block/sda6
[libfs_mgr]__mount(source=/dev/block/by-name/system_b,target=/mnt/scratch,type=f2fs)=0: Success
Using overlayfs for /system
. . .

Signed-off-by: Mark Salyzyn <salyzyn@google.com>
Test: adb-remount-test.sh
Change-Id: Ic8c642912b1bafe0b4210c69c99a1d89fa20f204
diff --git a/fs_mgr/fs_mgr_overlayfs.cpp b/fs_mgr/fs_mgr_overlayfs.cpp
index 1fa1aa1..a7704de 100644
--- a/fs_mgr/fs_mgr_overlayfs.cpp
+++ b/fs_mgr/fs_mgr_overlayfs.cpp
@@ -809,15 +809,26 @@
     entry.fs_type = mnt_type;
     if ((mnt_type == "f2fs") && !f2fs) entry.fs_type = "ext4";
     if ((mnt_type == "ext4") && !ext4) entry.fs_type = "f2fs";
-    entry.flags = MS_NOATIME;
-    if (readonly) {
-        entry.flags |= MS_RDONLY;
-    } else {
+    entry.flags = MS_NOATIME | MS_RDONLY;
+    auto mounted = true;
+    if (!readonly) {
+        if (entry.fs_type == "ext4") {
+            // check if ext4 de-dupe
+            entry.flags |= MS_RDONLY;
+            auto save_errno = errno;
+            mounted = fs_mgr_do_mount_one(entry) == 0;
+            if (mounted) {
+                mounted = !fs_mgr_has_shared_blocks(entry.mount_point, entry.blk_device);
+                fs_mgr_overlayfs_umount_scratch();
+            }
+            errno = save_errno;
+        }
+        entry.flags &= ~MS_RDONLY;
         fs_mgr_set_blk_ro(device_path, false);
     }
     entry.fs_mgr_flags.check = true;
     auto save_errno = errno;
-    auto mounted = fs_mgr_do_mount_one(entry) == 0;
+    if (mounted) mounted = fs_mgr_do_mount_one(entry) == 0;
     if (!mounted) {
         if ((entry.fs_type == "f2fs") && ext4) {
             entry.fs_type = "ext4";