soong_jar: Parallel compression
This compresses multiple files in parallel, and will split up larger
files (5MB+) into smaller chunks (1MB) to compress in parallel.
There is a small size overhead to recombine the chunks, but it's only a
few bytes per chunk, so for a 1MB chunk, it's minimal.
Rough numbers, with everything in the page cache, this can compress
~4GB (1000 files) down to 1GB in 6.5 seconds, instead of 120 seconds with
the non-parallel soong_jar and 150 seconds with zip.
Go's DEFLATE algorithm is still a bit worse than zip's -- about 3.5%
larger file sizes, but for most of our "dist" targets that is fine.
Change-Id: Ie4886c7d0f954ace46e599156e35fea7e74d6dd7
diff --git a/Android.bp b/Android.bp
index 523bec1..e287dab 100644
--- a/Android.bp
+++ b/Android.bp
@@ -206,8 +206,10 @@
blueprint_go_binary {
name: "soong_jar",
+ deps: ["android-archive-zip"],
srcs: [
"cmd/soong_jar/soong_jar.go",
+ "cmd/soong_jar/rate_limit.go",
],
}