Create Bazel symlink forest in a separate process.

This helps with incrementality a lot: the symlink forest must depend on
almost every directory in the source tree so that if a new file is added
or removed from *anywhere*, it is regenerated.

Previously, we couldn't do this without invoking bp2build, which is
quite wasteful because bp2build takes way more time than the symlink
forest creation, even though we do the latter in a very suboptimal way
at the moment.

This means that if a source file is added or removed (which does not
affect globs), we don't pay the cost of bp2build anymore.

Also refactored symlink_forest.go on the side. Too much state was being
passed around in arguments.

This change reimplements aosp/2263423 ; the semantics of not touching an
output file is the exact same as order-only inputs and the latter is a
bit fewer lines of code.

Test: Presubmits.
Change-Id: I565c580df8a01bacf175d56747c3f50743d4a4d4
diff --git a/tests/bootstrap_test.sh b/tests/bootstrap_test.sh
index e92a561..2331eb1 100755
--- a/tests/bootstrap_test.sh
+++ b/tests/bootstrap_test.sh
@@ -551,8 +551,45 @@
 
   run_soong bp2build
 
+  if [[ ! -f "./out/soong/bp2build_files_marker" ]]; then
+    fail "bp2build marker file was not generated"
+  fi
+
   if [[ ! -f "./out/soong/bp2build_workspace_marker" ]]; then
-    fail "Marker file was not generated"
+    fail "symlink forest marker file was not generated"
+  fi
+}
+
+function test_bp2build_add_irrelevant_file {
+  setup
+
+  mkdir -p a/b
+  touch a/b/c.txt
+  cat > a/b/Android.bp <<'EOF'
+filegroup {
+  name: "c",
+  srcs: ["c.txt"],
+  bazel_module: { bp2build_available: true },
+}
+EOF
+
+  run_soong bp2build
+  if [[ ! -e out/soong/bp2build/a/b/BUILD.bazel ]]; then
+    fail "BUILD file in symlink forest was not created";
+  fi
+
+  local mtime1=$(stat -c "%y" out/soong/bp2build/a/b/BUILD.bazel)
+
+  touch a/irrelevant.txt
+  run_soong bp2build
+  local mtime2=$(stat -c "%y" out/soong/bp2build/a/b/BUILD.bazel)
+
+  if [[ "$mtime1" != "$mtime2" ]]; then
+    fail "BUILD.bazel file was regenerated"
+  fi
+
+  if [[ ! -e "out/soong/workspace/a/irrelevant.txt" ]]; then
+    fail "New file was not symlinked into symlink forest"
   fi
 }
 
@@ -849,6 +886,7 @@
 test_bp2build_null_build
 test_bp2build_back_and_forth_null_build
 test_bp2build_add_android_bp
+test_bp2build_add_irrelevant_file
 test_bp2build_add_to_glob
 test_bp2build_bazel_workspace_structure
 test_bp2build_bazel_workspace_add_file