Bill Peckham | e9eb5f9 | 2019-02-01 15:52:10 -0800 | [diff] [blame] | 1 | #!/usr/bin/env python |
| 2 | # |
Daniel Norman | 0374741 | 2022-02-25 10:38:37 -0800 | [diff] [blame^] | 3 | # Copyright (C) 2022 The Android Open Source Project |
Bill Peckham | e9eb5f9 | 2019-02-01 15:52:10 -0800 | [diff] [blame] | 4 | # |
| 5 | # Licensed under the Apache License, Version 2.0 (the "License"); you may not |
| 6 | # use this file except in compliance with the License. You may obtain a copy of |
| 7 | # the License at |
| 8 | # |
| 9 | # http://www.apache.org/licenses/LICENSE-2.0 |
| 10 | # |
| 11 | # Unless required by applicable law or agreed to in writing, software |
| 12 | # distributed under the License is distributed on an "AS IS" BASIS, WITHOUT |
| 13 | # WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the |
| 14 | # License for the specific language governing permissions and limitations under |
| 15 | # the License. |
Daniel Norman | 4cc9df6 | 2019-07-18 10:11:07 -0700 | [diff] [blame] | 16 | # |
| 17 | """This script merges two partial target files packages. |
Bill Peckham | e9eb5f9 | 2019-02-01 15:52:10 -0800 | [diff] [blame] | 18 | |
Daniel Norman | dbbf5a3 | 2020-10-22 16:03:32 -0700 | [diff] [blame] | 19 | One input package contains framework files, and the other contains vendor files. |
Bill Peckham | e9eb5f9 | 2019-02-01 15:52:10 -0800 | [diff] [blame] | 20 | |
Daniel Norman | dbbf5a3 | 2020-10-22 16:03:32 -0700 | [diff] [blame] | 21 | This script produces a complete, merged target files package: |
| 22 | - This package can be used to generate a flashable IMG package. |
| 23 | See --output-img. |
| 24 | - This package can be used to generate an OTA package. See --output-ota. |
| 25 | - The merged package is checked for compatibility between the two inputs. |
| 26 | |
| 27 | Usage: merge_target_files [args] |
Bill Peckham | e9eb5f9 | 2019-02-01 15:52:10 -0800 | [diff] [blame] | 28 | |
Daniel Norman | d5d70ea | 2019-06-05 15:13:43 -0700 | [diff] [blame] | 29 | --framework-target-files framework-target-files-zip-archive |
| 30 | The input target files package containing framework bits. This is a zip |
Bill Peckham | e9eb5f9 | 2019-02-01 15:52:10 -0800 | [diff] [blame] | 31 | archive. |
| 32 | |
Daniel Norman | d5d70ea | 2019-06-05 15:13:43 -0700 | [diff] [blame] | 33 | --framework-item-list framework-item-list-file |
Daniel Norman | 2c99c5b | 2019-03-07 13:01:48 -0800 | [diff] [blame] | 34 | The optional path to a newline-separated config file that replaces the |
Daniel Norman | d5d70ea | 2019-06-05 15:13:43 -0700 | [diff] [blame] | 35 | contents of DEFAULT_FRAMEWORK_ITEM_LIST if provided. |
Daniel Norman | 2c99c5b | 2019-03-07 13:01:48 -0800 | [diff] [blame] | 36 | |
Daniel Norman | d5d70ea | 2019-06-05 15:13:43 -0700 | [diff] [blame] | 37 | --framework-misc-info-keys framework-misc-info-keys-file |
Daniel Norman | 2c99c5b | 2019-03-07 13:01:48 -0800 | [diff] [blame] | 38 | The optional path to a newline-separated config file that replaces the |
Daniel Norman | d5d70ea | 2019-06-05 15:13:43 -0700 | [diff] [blame] | 39 | contents of DEFAULT_FRAMEWORK_MISC_INFO_KEYS if provided. |
Daniel Norman | 2c99c5b | 2019-03-07 13:01:48 -0800 | [diff] [blame] | 40 | |
Daniel Norman | d5d70ea | 2019-06-05 15:13:43 -0700 | [diff] [blame] | 41 | --vendor-target-files vendor-target-files-zip-archive |
| 42 | The input target files package containing vendor bits. This is a zip |
Bill Peckham | e9eb5f9 | 2019-02-01 15:52:10 -0800 | [diff] [blame] | 43 | archive. |
| 44 | |
Daniel Norman | d5d70ea | 2019-06-05 15:13:43 -0700 | [diff] [blame] | 45 | --vendor-item-list vendor-item-list-file |
Daniel Norman | 2c99c5b | 2019-03-07 13:01:48 -0800 | [diff] [blame] | 46 | The optional path to a newline-separated config file that replaces the |
Daniel Norman | d5d70ea | 2019-06-05 15:13:43 -0700 | [diff] [blame] | 47 | contents of DEFAULT_VENDOR_ITEM_LIST if provided. |
Daniel Norman | 2c99c5b | 2019-03-07 13:01:48 -0800 | [diff] [blame] | 48 | |
Bill Peckham | e9eb5f9 | 2019-02-01 15:52:10 -0800 | [diff] [blame] | 49 | --output-target-files output-target-files-package |
Daniel Norman | fdb3881 | 2019-04-15 09:47:24 -0700 | [diff] [blame] | 50 | If provided, the output merged target files package. Also a zip archive. |
| 51 | |
| 52 | --output-dir output-directory |
| 53 | If provided, the destination directory for saving merged files. Requires |
| 54 | the --output-item-list flag. |
| 55 | Can be provided alongside --output-target-files, or by itself. |
| 56 | |
| 57 | --output-item-list output-item-list-file. |
| 58 | The optional path to a newline-separated config file that specifies the |
| 59 | file patterns to copy into the --output-dir. Required if providing |
| 60 | the --output-dir flag. |
Daniel Norman | a4911da | 2019-03-15 14:36:21 -0700 | [diff] [blame] | 61 | |
Daniel Norman | 3b64ce1 | 2019-04-16 16:11:35 -0700 | [diff] [blame] | 62 | --output-ota output-ota-package |
| 63 | The output ota package. This is a zip archive. Use of this flag may |
| 64 | require passing the --path common flag; see common.py. |
| 65 | |
Daniel Norman | 1bd2a1d | 2019-04-18 12:32:18 -0700 | [diff] [blame] | 66 | --output-img output-img-package |
| 67 | The output img package, suitable for use with 'fastboot update'. Use of |
| 68 | this flag may require passing the --path common flag; see common.py. |
| 69 | |
Daniel Norman | f031825 | 2019-04-15 11:34:56 -0700 | [diff] [blame] | 70 | --output-super-empty output-super-empty-image |
| 71 | If provided, creates a super_empty.img file from the merged target |
| 72 | files package and saves it at this path. |
| 73 | |
Daniel Norman | a4911da | 2019-03-15 14:36:21 -0700 | [diff] [blame] | 74 | --rebuild_recovery |
Po Hu | a6c5912 | 2022-02-16 08:41:29 +0000 | [diff] [blame] | 75 | Copy the recovery image used by non-A/B devices, used when |
| 76 | regenerating vendor images with --rebuild-sepolicy. |
Bill Peckham | 364c1cc | 2019-03-29 18:27:23 -0700 | [diff] [blame] | 77 | |
Daniel Norman | b0c7591 | 2020-09-24 14:30:21 -0700 | [diff] [blame] | 78 | --allow-duplicate-apkapex-keys |
| 79 | If provided, duplicate APK/APEX keys are ignored and the value from the |
| 80 | framework is used. |
| 81 | |
Daniel Norman | 571e182 | 2021-06-25 17:18:25 -0700 | [diff] [blame] | 82 | --rebuild-sepolicy |
| 83 | If provided, rebuilds odm.img or vendor.img to include merged sepolicy |
| 84 | files. If odm is present then odm is preferred. |
| 85 | |
| 86 | --vendor-otatools otatools.zip |
| 87 | If provided, use this otatools.zip when recompiling the odm or vendor |
| 88 | image to include sepolicy. |
| 89 | |
Bill Peckham | 364c1cc | 2019-03-29 18:27:23 -0700 | [diff] [blame] | 90 | --keep-tmp |
| 91 | Keep tempoary files for debugging purposes. |
Jose Galmes | 9c8f6eb | 2021-07-21 09:34:08 -0700 | [diff] [blame] | 92 | |
| 93 | The following only apply when using the VSDK to perform dexopt on vendor apps: |
| 94 | |
| 95 | --framework-dexpreopt-config |
| 96 | If provided, the location of framwework's dexpreopt_config.zip. |
| 97 | |
| 98 | --framework-dexpreopt-tools |
| 99 | if provided, the location of framework's dexpreopt_tools.zip. |
| 100 | |
| 101 | --vendor-dexpreopt-config |
| 102 | If provided, the location of vendor's dexpreopt_config.zip. |
Bill Peckham | e9eb5f9 | 2019-02-01 15:52:10 -0800 | [diff] [blame] | 103 | """ |
| 104 | |
Bill Peckham | e9eb5f9 | 2019-02-01 15:52:10 -0800 | [diff] [blame] | 105 | import fnmatch |
Jose Galmes | 9c8f6eb | 2021-07-21 09:34:08 -0700 | [diff] [blame] | 106 | import glob |
Daniel Norman | d335156 | 2020-10-29 12:33:11 -0700 | [diff] [blame] | 107 | import json |
Bill Peckham | e9eb5f9 | 2019-02-01 15:52:10 -0800 | [diff] [blame] | 108 | import logging |
| 109 | import os |
Bill Peckham | 19c3feb | 2020-03-20 18:31:43 -0700 | [diff] [blame] | 110 | import re |
Daniel Norman | fdb3881 | 2019-04-15 09:47:24 -0700 | [diff] [blame] | 111 | import shutil |
Bill Peckham | 540d91a | 2019-04-25 14:18:16 -0700 | [diff] [blame] | 112 | import subprocess |
Bill Peckham | e9eb5f9 | 2019-02-01 15:52:10 -0800 | [diff] [blame] | 113 | import sys |
| 114 | import zipfile |
Daniel Norman | 48603ff | 2021-02-22 15:15:24 -0800 | [diff] [blame] | 115 | from xml.etree import ElementTree |
Bill Peckham | e9eb5f9 | 2019-02-01 15:52:10 -0800 | [diff] [blame] | 116 | |
Bill Peckham | e9eb5f9 | 2019-02-01 15:52:10 -0800 | [diff] [blame] | 117 | import add_img_to_target_files |
Daniel Norman | e9af70a | 2021-04-15 16:39:22 -0700 | [diff] [blame] | 118 | import apex_utils |
Daniel Norman | db8cacc | 2021-04-09 15:34:43 -0700 | [diff] [blame] | 119 | import build_image |
Daniel Norman | f031825 | 2019-04-15 11:34:56 -0700 | [diff] [blame] | 120 | import build_super_image |
Yifan Hong | ade0d3f | 2019-08-21 16:37:11 -0700 | [diff] [blame] | 121 | import check_target_files_vintf |
Daniel Norman | f031825 | 2019-04-15 11:34:56 -0700 | [diff] [blame] | 122 | import common |
Daniel Norman | 1bd2a1d | 2019-04-18 12:32:18 -0700 | [diff] [blame] | 123 | import img_from_target_files |
Daniel Norman | b8d52a2 | 2020-10-26 17:55:00 -0700 | [diff] [blame] | 124 | import find_shareduid_violation |
Daniel Norman | 3b64ce1 | 2019-04-16 16:11:35 -0700 | [diff] [blame] | 125 | import ota_from_target_files |
Daniel Norman | db8cacc | 2021-04-09 15:34:43 -0700 | [diff] [blame] | 126 | import sparse_img |
| 127 | import verity_utils |
| 128 | |
Daniel Norman | a84d13b | 2022-02-17 14:16:40 -0800 | [diff] [blame] | 129 | from common import ExternalError |
Bill Peckham | e9eb5f9 | 2019-02-01 15:52:10 -0800 | [diff] [blame] | 130 | |
| 131 | logger = logging.getLogger(__name__) |
Tao Bao | 2ad4b82 | 2019-06-27 16:52:12 -0700 | [diff] [blame] | 132 | |
Bill Peckham | e9eb5f9 | 2019-02-01 15:52:10 -0800 | [diff] [blame] | 133 | OPTIONS = common.OPTIONS |
Bill Peckham | cb84817 | 2020-04-03 12:50:47 -0700 | [diff] [blame] | 134 | # Always turn on verbose logging. |
| 135 | OPTIONS.verbose = True |
Daniel Norman | d5d70ea | 2019-06-05 15:13:43 -0700 | [diff] [blame] | 136 | OPTIONS.framework_target_files = None |
| 137 | OPTIONS.framework_item_list = None |
| 138 | OPTIONS.framework_misc_info_keys = None |
| 139 | OPTIONS.vendor_target_files = None |
| 140 | OPTIONS.vendor_item_list = None |
Bill Peckham | f753e15 | 2019-02-19 18:02:46 -0800 | [diff] [blame] | 141 | OPTIONS.output_target_files = None |
Daniel Norman | fdb3881 | 2019-04-15 09:47:24 -0700 | [diff] [blame] | 142 | OPTIONS.output_dir = None |
| 143 | OPTIONS.output_item_list = None |
Daniel Norman | 3b64ce1 | 2019-04-16 16:11:35 -0700 | [diff] [blame] | 144 | OPTIONS.output_ota = None |
Daniel Norman | 1bd2a1d | 2019-04-18 12:32:18 -0700 | [diff] [blame] | 145 | OPTIONS.output_img = None |
Daniel Norman | f031825 | 2019-04-15 11:34:56 -0700 | [diff] [blame] | 146 | OPTIONS.output_super_empty = None |
Daniel Norman | a4911da | 2019-03-15 14:36:21 -0700 | [diff] [blame] | 147 | OPTIONS.rebuild_recovery = False |
Daniel Norman | b0c7591 | 2020-09-24 14:30:21 -0700 | [diff] [blame] | 148 | # TODO(b/150582573): Remove this option. |
| 149 | OPTIONS.allow_duplicate_apkapex_keys = False |
Daniel Norman | 571e182 | 2021-06-25 17:18:25 -0700 | [diff] [blame] | 150 | OPTIONS.vendor_otatools = None |
| 151 | OPTIONS.rebuild_sepolicy = False |
Bill Peckham | f753e15 | 2019-02-19 18:02:46 -0800 | [diff] [blame] | 152 | OPTIONS.keep_tmp = False |
Jose Galmes | 9c8f6eb | 2021-07-21 09:34:08 -0700 | [diff] [blame] | 153 | OPTIONS.framework_dexpreopt_config = None |
| 154 | OPTIONS.framework_dexpreopt_tools = None |
| 155 | OPTIONS.vendor_dexpreopt_config = None |
Bill Peckham | e9eb5f9 | 2019-02-01 15:52:10 -0800 | [diff] [blame] | 156 | |
Bill Peckham | 19c3feb | 2020-03-20 18:31:43 -0700 | [diff] [blame] | 157 | # In an item list (framework or vendor), we may see entries that select whole |
| 158 | # partitions. Such an entry might look like this 'SYSTEM/*' (e.g., for the |
| 159 | # system partition). The following regex matches this and extracts the |
| 160 | # partition name. |
| 161 | |
| 162 | PARTITION_ITEM_PATTERN = re.compile(r'^([A-Z_]+)/\*$') |
| 163 | |
Bill Peckham | 5c7b034 | 2020-04-03 15:36:23 -0700 | [diff] [blame] | 164 | # In apexkeys.txt or apkcerts.txt, we will find partition tags on each entry in |
| 165 | # the file. We use these partition tags to filter the entries in those files |
| 166 | # from the two different target files packages to produce a merged apexkeys.txt |
| 167 | # or apkcerts.txt file. A partition tag (e.g., for the product partition) looks |
| 168 | # like this: 'partition="product"'. We use the group syntax grab the value of |
| 169 | # the tag. We use non-greedy matching in case there are other fields on the |
| 170 | # same line. |
Bill Peckham | 19c3feb | 2020-03-20 18:31:43 -0700 | [diff] [blame] | 171 | |
Bill Peckham | 5c7b034 | 2020-04-03 15:36:23 -0700 | [diff] [blame] | 172 | PARTITION_TAG_PATTERN = re.compile(r'partition="(.*?)"') |
Bill Peckham | 19c3feb | 2020-03-20 18:31:43 -0700 | [diff] [blame] | 173 | |
| 174 | # The sorting algorithm for apexkeys.txt and apkcerts.txt does not include the |
| 175 | # ".apex" or ".apk" suffix, so we use the following pattern to extract a key. |
| 176 | |
| 177 | MODULE_KEY_PATTERN = re.compile(r'name="(.+)\.(apex|apk)"') |
| 178 | |
Daniel Norman | d5d70ea | 2019-06-05 15:13:43 -0700 | [diff] [blame] | 179 | # DEFAULT_FRAMEWORK_ITEM_LIST is a list of items to extract from the partial |
| 180 | # framework target files package as is, meaning these items will land in the |
Bill Peckham | e9eb5f9 | 2019-02-01 15:52:10 -0800 | [diff] [blame] | 181 | # output target files package exactly as they appear in the input partial |
Daniel Norman | d5d70ea | 2019-06-05 15:13:43 -0700 | [diff] [blame] | 182 | # framework target files package. |
Bill Peckham | e9eb5f9 | 2019-02-01 15:52:10 -0800 | [diff] [blame] | 183 | |
Daniel Norman | d5d70ea | 2019-06-05 15:13:43 -0700 | [diff] [blame] | 184 | DEFAULT_FRAMEWORK_ITEM_LIST = ( |
Bill Peckham | e9eb5f9 | 2019-02-01 15:52:10 -0800 | [diff] [blame] | 185 | 'META/apkcerts.txt', |
| 186 | 'META/filesystem_config.txt', |
| 187 | 'META/root_filesystem_config.txt', |
Bill Peckham | e9eb5f9 | 2019-02-01 15:52:10 -0800 | [diff] [blame] | 188 | 'META/update_engine_config.txt', |
| 189 | 'PRODUCT/*', |
| 190 | 'ROOT/*', |
| 191 | 'SYSTEM/*', |
Daniel Norman | edf1247 | 2019-05-22 10:47:08 -0700 | [diff] [blame] | 192 | ) |
Bill Peckham | e9eb5f9 | 2019-02-01 15:52:10 -0800 | [diff] [blame] | 193 | |
Daniel Norman | d5d70ea | 2019-06-05 15:13:43 -0700 | [diff] [blame] | 194 | # DEFAULT_FRAMEWORK_MISC_INFO_KEYS is a list of keys to obtain from the |
Daniel Norman | dbbf5a3 | 2020-10-22 16:03:32 -0700 | [diff] [blame] | 195 | # framework instance of META/misc_info.txt. The remaining keys should come |
| 196 | # from the vendor instance. |
Bill Peckham | e9eb5f9 | 2019-02-01 15:52:10 -0800 | [diff] [blame] | 197 | |
Daniel Norman | d5d70ea | 2019-06-05 15:13:43 -0700 | [diff] [blame] | 198 | DEFAULT_FRAMEWORK_MISC_INFO_KEYS = ( |
Bill Peckham | e9eb5f9 | 2019-02-01 15:52:10 -0800 | [diff] [blame] | 199 | 'avb_system_hashtree_enable', |
| 200 | 'avb_system_add_hashtree_footer_args', |
| 201 | 'avb_system_key_path', |
| 202 | 'avb_system_algorithm', |
| 203 | 'avb_system_rollback_index_location', |
| 204 | 'avb_product_hashtree_enable', |
| 205 | 'avb_product_add_hashtree_footer_args', |
Justin Yun | 6151e3f | 2019-06-25 15:58:13 +0900 | [diff] [blame] | 206 | 'avb_system_ext_hashtree_enable', |
| 207 | 'avb_system_ext_add_hashtree_footer_args', |
Bill Peckham | e9eb5f9 | 2019-02-01 15:52:10 -0800 | [diff] [blame] | 208 | 'system_root_image', |
| 209 | 'root_dir', |
| 210 | 'ab_update', |
| 211 | 'default_system_dev_certificate', |
| 212 | 'system_size', |
Chris Gross | 203191b | 2020-05-30 02:39:12 +0000 | [diff] [blame] | 213 | 'building_system_image', |
| 214 | 'building_system_ext_image', |
| 215 | 'building_product_image', |
Daniel Norman | edf1247 | 2019-05-22 10:47:08 -0700 | [diff] [blame] | 216 | ) |
Bill Peckham | e9eb5f9 | 2019-02-01 15:52:10 -0800 | [diff] [blame] | 217 | |
Daniel Norman | d5d70ea | 2019-06-05 15:13:43 -0700 | [diff] [blame] | 218 | # DEFAULT_VENDOR_ITEM_LIST is a list of items to extract from the partial |
| 219 | # vendor target files package as is, meaning these items will land in the output |
| 220 | # target files package exactly as they appear in the input partial vendor target |
Bill Peckham | e9eb5f9 | 2019-02-01 15:52:10 -0800 | [diff] [blame] | 221 | # files package. |
| 222 | |
Daniel Norman | d5d70ea | 2019-06-05 15:13:43 -0700 | [diff] [blame] | 223 | DEFAULT_VENDOR_ITEM_LIST = ( |
Bill Peckham | e9eb5f9 | 2019-02-01 15:52:10 -0800 | [diff] [blame] | 224 | 'META/boot_filesystem_config.txt', |
| 225 | 'META/otakeys.txt', |
| 226 | 'META/releasetools.py', |
| 227 | 'META/vendor_filesystem_config.txt', |
Bill Peckham | e9eb5f9 | 2019-02-01 15:52:10 -0800 | [diff] [blame] | 228 | 'BOOT/*', |
| 229 | 'DATA/*', |
| 230 | 'ODM/*', |
| 231 | 'OTA/android-info.txt', |
| 232 | 'PREBUILT_IMAGES/*', |
| 233 | 'RADIO/*', |
| 234 | 'VENDOR/*', |
Daniel Norman | edf1247 | 2019-05-22 10:47:08 -0700 | [diff] [blame] | 235 | ) |
Bill Peckham | e9eb5f9 | 2019-02-01 15:52:10 -0800 | [diff] [blame] | 236 | |
Daniel Norman | edf1247 | 2019-05-22 10:47:08 -0700 | [diff] [blame] | 237 | # The merge config lists should not attempt to extract items from both |
| 238 | # builds for any of the following partitions. The partitions in |
| 239 | # SINGLE_BUILD_PARTITIONS should come entirely from a single build (either |
Daniel Norman | d5d70ea | 2019-06-05 15:13:43 -0700 | [diff] [blame] | 240 | # framework or vendor, but not both). |
Daniel Norman | edf1247 | 2019-05-22 10:47:08 -0700 | [diff] [blame] | 241 | |
| 242 | SINGLE_BUILD_PARTITIONS = ( |
| 243 | 'BOOT/', |
| 244 | 'DATA/', |
| 245 | 'ODM/', |
| 246 | 'PRODUCT/', |
Justin Yun | 6151e3f | 2019-06-25 15:58:13 +0900 | [diff] [blame] | 247 | 'SYSTEM_EXT/', |
Daniel Norman | edf1247 | 2019-05-22 10:47:08 -0700 | [diff] [blame] | 248 | 'RADIO/', |
| 249 | 'RECOVERY/', |
| 250 | 'ROOT/', |
| 251 | 'SYSTEM/', |
| 252 | 'SYSTEM_OTHER/', |
| 253 | 'VENDOR/', |
Yifan Hong | cfb917a | 2020-05-07 14:58:20 -0700 | [diff] [blame] | 254 | 'VENDOR_DLKM/', |
Yifan Hong | f496f1b | 2020-07-15 16:52:59 -0700 | [diff] [blame] | 255 | 'ODM_DLKM/', |
Ramji Jiyani | 13a4137 | 2022-01-27 07:05:08 +0000 | [diff] [blame] | 256 | 'SYSTEM_DLKM/', |
Daniel Norman | edf1247 | 2019-05-22 10:47:08 -0700 | [diff] [blame] | 257 | ) |
Bill Peckham | e9eb5f9 | 2019-02-01 15:52:10 -0800 | [diff] [blame] | 258 | |
| 259 | |
Chris Gross | fabf50a | 2019-05-02 12:42:09 -0700 | [diff] [blame] | 260 | def write_sorted_data(data, path): |
Tao Bao | 2ad4b82 | 2019-06-27 16:52:12 -0700 | [diff] [blame] | 261 | """Writes the sorted contents of either a list or dict to file. |
Chris Gross | fabf50a | 2019-05-02 12:42:09 -0700 | [diff] [blame] | 262 | |
Tao Bao | 2ad4b82 | 2019-06-27 16:52:12 -0700 | [diff] [blame] | 263 | This function sorts the contents of the list or dict and then writes the |
| 264 | resulting sorted contents to a file specified by path. |
Chris Gross | fabf50a | 2019-05-02 12:42:09 -0700 | [diff] [blame] | 265 | |
| 266 | Args: |
| 267 | data: The list or dict to sort and write. |
| 268 | path: Path to the file to write the sorted values to. The file at path will |
| 269 | be overridden if it exists. |
| 270 | """ |
| 271 | with open(path, 'w') as output: |
Daniel Norman | d5d70ea | 2019-06-05 15:13:43 -0700 | [diff] [blame] | 272 | for entry in sorted(data): |
Chris Gross | fabf50a | 2019-05-02 12:42:09 -0700 | [diff] [blame] | 273 | out_str = '{}={}\n'.format(entry, data[entry]) if isinstance( |
| 274 | data, dict) else '{}\n'.format(entry) |
| 275 | output.write(out_str) |
| 276 | |
| 277 | |
Daniel Norman | 0374741 | 2022-02-25 10:38:37 -0800 | [diff] [blame^] | 278 | def extract_items(input_zip, output_dir, extract_item_list): |
| 279 | """Extracts items in extra_item_list from a zip to a dir.""" |
Bill Peckham | e9eb5f9 | 2019-02-01 15:52:10 -0800 | [diff] [blame] | 280 | |
Daniel Norman | 0374741 | 2022-02-25 10:38:37 -0800 | [diff] [blame^] | 281 | logger.info('extracting from %s', input_zip) |
Bill Peckham | e9eb5f9 | 2019-02-01 15:52:10 -0800 | [diff] [blame] | 282 | |
| 283 | # Filter the extract_item_list to remove any items that do not exist in the |
| 284 | # zip file. Otherwise, the extraction step will fail. |
| 285 | |
Daniel Norman | 0374741 | 2022-02-25 10:38:37 -0800 | [diff] [blame^] | 286 | with zipfile.ZipFile(input_zip, allowZip64=True) as input_zipfile: |
| 287 | input_namelist = input_zipfile.namelist() |
Bill Peckham | e9eb5f9 | 2019-02-01 15:52:10 -0800 | [diff] [blame] | 288 | |
| 289 | filtered_extract_item_list = [] |
| 290 | for pattern in extract_item_list: |
Daniel Norman | 0374741 | 2022-02-25 10:38:37 -0800 | [diff] [blame^] | 291 | matching_namelist = fnmatch.filter(input_namelist, pattern) |
Bill Peckham | e9eb5f9 | 2019-02-01 15:52:10 -0800 | [diff] [blame] | 292 | if not matching_namelist: |
| 293 | logger.warning('no match for %s', pattern) |
| 294 | else: |
| 295 | filtered_extract_item_list.append(pattern) |
| 296 | |
Daniel Norman | 0374741 | 2022-02-25 10:38:37 -0800 | [diff] [blame^] | 297 | common.UnzipToDir(input_zip, output_dir, filtered_extract_item_list) |
Bill Peckham | e9eb5f9 | 2019-02-01 15:52:10 -0800 | [diff] [blame] | 298 | |
| 299 | |
Daniel Norman | fdb3881 | 2019-04-15 09:47:24 -0700 | [diff] [blame] | 300 | def copy_items(from_dir, to_dir, patterns): |
| 301 | """Similar to extract_items() except uses an input dir instead of zip.""" |
| 302 | file_paths = [] |
| 303 | for dirpath, _, filenames in os.walk(from_dir): |
Daniel Norman | e5b134a | 2019-04-17 14:54:06 -0700 | [diff] [blame] | 304 | file_paths.extend( |
| 305 | os.path.relpath(path=os.path.join(dirpath, filename), start=from_dir) |
| 306 | for filename in filenames) |
Daniel Norman | fdb3881 | 2019-04-15 09:47:24 -0700 | [diff] [blame] | 307 | |
| 308 | filtered_file_paths = set() |
| 309 | for pattern in patterns: |
| 310 | filtered_file_paths.update(fnmatch.filter(file_paths, pattern)) |
| 311 | |
| 312 | for file_path in filtered_file_paths: |
| 313 | original_file_path = os.path.join(from_dir, file_path) |
| 314 | copied_file_path = os.path.join(to_dir, file_path) |
| 315 | copied_file_dir = os.path.dirname(copied_file_path) |
| 316 | if not os.path.exists(copied_file_dir): |
| 317 | os.makedirs(copied_file_dir) |
| 318 | if os.path.islink(original_file_path): |
| 319 | os.symlink(os.readlink(original_file_path), copied_file_path) |
| 320 | else: |
| 321 | shutil.copyfile(original_file_path, copied_file_path) |
| 322 | |
| 323 | |
Daniel Norman | 0374741 | 2022-02-25 10:38:37 -0800 | [diff] [blame^] | 324 | def validate_config_lists(): |
Daniel Norman | e596452 | 2019-03-19 10:32:03 -0700 | [diff] [blame] | 325 | """Performs validations on the merge config lists. |
| 326 | |
Daniel Norman | e596452 | 2019-03-19 10:32:03 -0700 | [diff] [blame] | 327 | Returns: |
| 328 | False if a validation fails, otherwise true. |
| 329 | """ |
Daniel Norman | edf1247 | 2019-05-22 10:47:08 -0700 | [diff] [blame] | 330 | has_error = False |
| 331 | |
Daniel Norman | d5d70ea | 2019-06-05 15:13:43 -0700 | [diff] [blame] | 332 | default_combined_item_set = set(DEFAULT_FRAMEWORK_ITEM_LIST) |
| 333 | default_combined_item_set.update(DEFAULT_VENDOR_ITEM_LIST) |
Daniel Norman | e596452 | 2019-03-19 10:32:03 -0700 | [diff] [blame] | 334 | |
Daniel Norman | 0374741 | 2022-02-25 10:38:37 -0800 | [diff] [blame^] | 335 | combined_item_set = set(OPTIONS.framework_item_list) |
| 336 | combined_item_set.update(OPTIONS.vendor_item_list) |
Daniel Norman | e596452 | 2019-03-19 10:32:03 -0700 | [diff] [blame] | 337 | |
| 338 | # Check that the merge config lists are not missing any item specified |
| 339 | # by the default config lists. |
| 340 | difference = default_combined_item_set.difference(combined_item_set) |
| 341 | if difference: |
Daniel Norman | e5b134a | 2019-04-17 14:54:06 -0700 | [diff] [blame] | 342 | logger.error('Missing merge config items: %s', list(difference)) |
Daniel Norman | e596452 | 2019-03-19 10:32:03 -0700 | [diff] [blame] | 343 | logger.error('Please ensure missing items are in either the ' |
Daniel Norman | d5d70ea | 2019-06-05 15:13:43 -0700 | [diff] [blame] | 344 | 'framework-item-list or vendor-item-list files provided to ' |
Daniel Norman | e596452 | 2019-03-19 10:32:03 -0700 | [diff] [blame] | 345 | 'this script.') |
Daniel Norman | edf1247 | 2019-05-22 10:47:08 -0700 | [diff] [blame] | 346 | has_error = True |
| 347 | |
Daniel Norman | dbbf5a3 | 2020-10-22 16:03:32 -0700 | [diff] [blame] | 348 | # Check that partitions only come from one input. |
Daniel Norman | edf1247 | 2019-05-22 10:47:08 -0700 | [diff] [blame] | 349 | for partition in SINGLE_BUILD_PARTITIONS: |
Daniel Norman | dbbf5a3 | 2020-10-22 16:03:32 -0700 | [diff] [blame] | 350 | image_path = 'IMAGES/{}.img'.format(partition.lower().replace('/', '')) |
| 351 | in_framework = ( |
Daniel Norman | 0374741 | 2022-02-25 10:38:37 -0800 | [diff] [blame^] | 352 | any(item.startswith(partition) for item in OPTIONS.framework_item_list) |
| 353 | or image_path in OPTIONS.framework_item_list) |
Daniel Norman | dbbf5a3 | 2020-10-22 16:03:32 -0700 | [diff] [blame] | 354 | in_vendor = ( |
Daniel Norman | 0374741 | 2022-02-25 10:38:37 -0800 | [diff] [blame^] | 355 | any(item.startswith(partition) for item in OPTIONS.vendor_item_list) or |
| 356 | image_path in OPTIONS.vendor_item_list) |
Daniel Norman | d5d70ea | 2019-06-05 15:13:43 -0700 | [diff] [blame] | 357 | if in_framework and in_vendor: |
Daniel Norman | edf1247 | 2019-05-22 10:47:08 -0700 | [diff] [blame] | 358 | logger.error( |
Tao Bao | 2ad4b82 | 2019-06-27 16:52:12 -0700 | [diff] [blame] | 359 | 'Cannot extract items from %s for both the framework and vendor' |
Kiyoung Kim | ebe7c9c | 2019-06-25 17:09:55 +0900 | [diff] [blame] | 360 | ' builds. Please ensure only one merge config item list' |
Tao Bao | 2ad4b82 | 2019-06-27 16:52:12 -0700 | [diff] [blame] | 361 | ' includes %s.', partition, partition) |
Daniel Norman | edf1247 | 2019-05-22 10:47:08 -0700 | [diff] [blame] | 362 | has_error = True |
Daniel Norman | e596452 | 2019-03-19 10:32:03 -0700 | [diff] [blame] | 363 | |
Daniel Norman | 0374741 | 2022-02-25 10:38:37 -0800 | [diff] [blame^] | 364 | if ('dynamic_partition_list' in OPTIONS.framework_misc_info_keys) or ( |
| 365 | 'super_partition_groups' in OPTIONS.framework_misc_info_keys): |
Daniel Norman | 19b9fe9 | 2019-03-19 14:48:02 -0700 | [diff] [blame] | 366 | logger.error('Dynamic partition misc info keys should come from ' |
Daniel Norman | d5d70ea | 2019-06-05 15:13:43 -0700 | [diff] [blame] | 367 | 'the vendor instance of META/misc_info.txt.') |
Daniel Norman | edf1247 | 2019-05-22 10:47:08 -0700 | [diff] [blame] | 368 | has_error = True |
Daniel Norman | 19b9fe9 | 2019-03-19 14:48:02 -0700 | [diff] [blame] | 369 | |
Daniel Norman | edf1247 | 2019-05-22 10:47:08 -0700 | [diff] [blame] | 370 | return not has_error |
Daniel Norman | e596452 | 2019-03-19 10:32:03 -0700 | [diff] [blame] | 371 | |
| 372 | |
Daniel Norman | 0374741 | 2022-02-25 10:38:37 -0800 | [diff] [blame^] | 373 | def merge_ab_partitions_txt(framework_meta_dir, vendor_meta_dir, |
| 374 | merged_meta_dir): |
| 375 | """Merges META/ab_partitions.txt. |
Bill Peckham | e9eb5f9 | 2019-02-01 15:52:10 -0800 | [diff] [blame] | 376 | |
Daniel Norman | 0374741 | 2022-02-25 10:38:37 -0800 | [diff] [blame^] | 377 | The output contains the union of the partition names. |
Bill Peckham | e9eb5f9 | 2019-02-01 15:52:10 -0800 | [diff] [blame] | 378 | """ |
Daniel Norman | 0374741 | 2022-02-25 10:38:37 -0800 | [diff] [blame^] | 379 | with open(os.path.join(framework_meta_dir, 'ab_partitions.txt')) as f: |
Daniel Norman | d5d70ea | 2019-06-05 15:13:43 -0700 | [diff] [blame] | 380 | framework_ab_partitions = f.read().splitlines() |
Bill Peckham | e9eb5f9 | 2019-02-01 15:52:10 -0800 | [diff] [blame] | 381 | |
Daniel Norman | 0374741 | 2022-02-25 10:38:37 -0800 | [diff] [blame^] | 382 | with open(os.path.join(vendor_meta_dir, 'ab_partitions.txt')) as f: |
Daniel Norman | d5d70ea | 2019-06-05 15:13:43 -0700 | [diff] [blame] | 383 | vendor_ab_partitions = f.read().splitlines() |
Bill Peckham | e9eb5f9 | 2019-02-01 15:52:10 -0800 | [diff] [blame] | 384 | |
Daniel Norman | 0374741 | 2022-02-25 10:38:37 -0800 | [diff] [blame^] | 385 | write_sorted_data( |
| 386 | data=set(framework_ab_partitions + vendor_ab_partitions), |
| 387 | path=os.path.join(merged_meta_dir, 'ab_partitions.txt')) |
Bill Peckham | e9eb5f9 | 2019-02-01 15:52:10 -0800 | [diff] [blame] | 388 | |
| 389 | |
Daniel Norman | 0374741 | 2022-02-25 10:38:37 -0800 | [diff] [blame^] | 390 | def merge_misc_info_txt(framework_meta_dir, vendor_meta_dir, merged_meta_dir): |
| 391 | """Merges META/misc_info.txt. |
Bill Peckham | e9eb5f9 | 2019-02-01 15:52:10 -0800 | [diff] [blame] | 392 | |
Daniel Norman | 0374741 | 2022-02-25 10:38:37 -0800 | [diff] [blame^] | 393 | The output contains a combination of key=value pairs from both inputs. |
| 394 | Most pairs are taken from the vendor input, while some are taken from |
| 395 | the framework input. |
Bill Peckham | e9eb5f9 | 2019-02-01 15:52:10 -0800 | [diff] [blame] | 396 | """ |
| 397 | |
Daniel Norman | 0374741 | 2022-02-25 10:38:37 -0800 | [diff] [blame^] | 398 | OPTIONS.framework_misc_info = common.LoadDictionaryFromFile( |
| 399 | os.path.join(framework_meta_dir, 'misc_info.txt')) |
| 400 | OPTIONS.vendor_misc_info = common.LoadDictionaryFromFile( |
| 401 | os.path.join(vendor_meta_dir, 'misc_info.txt')) |
Bill Peckham | e9eb5f9 | 2019-02-01 15:52:10 -0800 | [diff] [blame] | 402 | |
Daniel Norman | 0374741 | 2022-02-25 10:38:37 -0800 | [diff] [blame^] | 403 | # Merged misc info is a combination of vendor misc info plus certain values |
| 404 | # from the framework misc info. |
Bill Peckham | e9eb5f9 | 2019-02-01 15:52:10 -0800 | [diff] [blame] | 405 | |
Daniel Norman | 0374741 | 2022-02-25 10:38:37 -0800 | [diff] [blame^] | 406 | merged_dict = OPTIONS.vendor_misc_info |
| 407 | for key in OPTIONS.framework_misc_info_keys: |
| 408 | merged_dict[key] = OPTIONS.framework_misc_info[key] |
Daniel Norman | 19b9fe9 | 2019-03-19 14:48:02 -0700 | [diff] [blame] | 409 | |
Daniel Norman | 38888d3 | 2020-11-19 14:51:15 -0800 | [diff] [blame] | 410 | # If AVB is enabled then ensure that we build vbmeta.img. |
| 411 | # Partial builds with AVB enabled may set PRODUCT_BUILD_VBMETA_IMAGE=false to |
| 412 | # skip building an incomplete vbmeta.img. |
| 413 | if merged_dict.get('avb_enable') == 'true': |
| 414 | merged_dict['avb_building_vbmeta_image'] = 'true' |
| 415 | |
Daniel Norman | 0374741 | 2022-02-25 10:38:37 -0800 | [diff] [blame^] | 416 | return merged_dict |
Bill Peckham | e9eb5f9 | 2019-02-01 15:52:10 -0800 | [diff] [blame] | 417 | |
| 418 | |
Daniel Norman | 0374741 | 2022-02-25 10:38:37 -0800 | [diff] [blame^] | 419 | def merge_dynamic_partitions_info_txt(framework_meta_dir, vendor_meta_dir, |
| 420 | merged_meta_dir): |
| 421 | """Merge META/dynamic_partitions_info.txt.""" |
Kiyoung Kim | ebe7c9c | 2019-06-25 17:09:55 +0900 | [diff] [blame] | 422 | framework_dynamic_partitions_dict = common.LoadDictionaryFromFile( |
Daniel Norman | 0374741 | 2022-02-25 10:38:37 -0800 | [diff] [blame^] | 423 | os.path.join(framework_meta_dir, 'dynamic_partitions_info.txt')) |
Kiyoung Kim | ebe7c9c | 2019-06-25 17:09:55 +0900 | [diff] [blame] | 424 | vendor_dynamic_partitions_dict = common.LoadDictionaryFromFile( |
Daniel Norman | 0374741 | 2022-02-25 10:38:37 -0800 | [diff] [blame^] | 425 | os.path.join(vendor_meta_dir, 'dynamic_partitions_info.txt')) |
Daniel Norman | a61cde0 | 2019-05-03 14:19:13 -0700 | [diff] [blame] | 426 | |
Daniel Norman | bfc51ef | 2019-07-24 14:34:54 -0700 | [diff] [blame] | 427 | merged_dynamic_partitions_dict = common.MergeDynamicPartitionInfoDicts( |
Daniel Norman | d5d70ea | 2019-06-05 15:13:43 -0700 | [diff] [blame] | 428 | framework_dict=framework_dynamic_partitions_dict, |
Daniel Norman | 5541714 | 2019-11-25 16:04:36 -0800 | [diff] [blame] | 429 | vendor_dict=vendor_dynamic_partitions_dict) |
Daniel Norman | a61cde0 | 2019-05-03 14:19:13 -0700 | [diff] [blame] | 430 | |
Chris Gross | fabf50a | 2019-05-02 12:42:09 -0700 | [diff] [blame] | 431 | write_sorted_data( |
| 432 | data=merged_dynamic_partitions_dict, |
Daniel Norman | 0374741 | 2022-02-25 10:38:37 -0800 | [diff] [blame^] | 433 | path=os.path.join(merged_meta_dir, 'dynamic_partitions_info.txt')) |
| 434 | |
| 435 | # Merge misc info keys used for Dynamic Partitions. |
| 436 | OPTIONS.merged_misc_info.update(merged_dynamic_partitions_dict) |
| 437 | # Ensure that add_img_to_target_files rebuilds super split images for |
| 438 | # devices that retrofit dynamic partitions. This flag may have been set to |
| 439 | # false in the partial builds to prevent duplicate building of super.img. |
| 440 | OPTIONS.merged_misc_info['build_super_partition'] = 'true' |
Chris Gross | fabf50a | 2019-05-02 12:42:09 -0700 | [diff] [blame] | 441 | |
| 442 | |
Bill Peckham | 19c3feb | 2020-03-20 18:31:43 -0700 | [diff] [blame] | 443 | def item_list_to_partition_set(item_list): |
| 444 | """Converts a target files item list to a partition set. |
| 445 | |
| 446 | The item list contains items that might look like 'SYSTEM/*' or 'VENDOR/*' or |
| 447 | 'OTA/android-info.txt'. Items that end in '/*' are assumed to match entire |
| 448 | directories where 'SYSTEM' or 'VENDOR' is a directory name that identifies the |
| 449 | contents of a partition of the same name. Other items in the list, such as the |
| 450 | 'OTA' example contain metadata. This function iterates such a list, returning |
| 451 | a set that contains the partition entries. |
| 452 | |
| 453 | Args: |
| 454 | item_list: A list of items in a target files package. |
Daniel Norman | b0c7591 | 2020-09-24 14:30:21 -0700 | [diff] [blame] | 455 | |
Bill Peckham | 19c3feb | 2020-03-20 18:31:43 -0700 | [diff] [blame] | 456 | Returns: |
| 457 | A set of partitions extracted from the list of items. |
| 458 | """ |
| 459 | |
| 460 | partition_set = set() |
| 461 | |
| 462 | for item in item_list: |
| 463 | match = PARTITION_ITEM_PATTERN.search(item.strip()) |
| 464 | partition_tag = match.group(1).lower() if match else None |
| 465 | |
| 466 | if partition_tag: |
| 467 | partition_set.add(partition_tag) |
| 468 | |
| 469 | return partition_set |
| 470 | |
| 471 | |
Daniel Norman | 0374741 | 2022-02-25 10:38:37 -0800 | [diff] [blame^] | 472 | def merge_package_keys_txt(framework_meta_dir, vendor_meta_dir, merged_meta_dir, |
| 473 | file_name): |
| 474 | """Merges APK/APEX key list files.""" |
Chris Gross | fabf50a | 2019-05-02 12:42:09 -0700 | [diff] [blame] | 475 | |
Daniel Norman | 0374741 | 2022-02-25 10:38:37 -0800 | [diff] [blame^] | 476 | if file_name not in ('apkcerts.txt', 'apexkeys.txt'): |
| 477 | raise ExternalError( |
| 478 | 'Unexpected file_name provided to merge_package_keys_txt: %s', |
| 479 | file_name) |
Chris Gross | fabf50a | 2019-05-02 12:42:09 -0700 | [diff] [blame] | 480 | |
| 481 | def read_helper(d): |
| 482 | temp = {} |
Daniel Norman | 0374741 | 2022-02-25 10:38:37 -0800 | [diff] [blame^] | 483 | with open(os.path.join(d, file_name)) as f: |
| 484 | for line in f.read().splitlines(): |
| 485 | line = line.strip() |
| 486 | if line: |
| 487 | name_search = MODULE_KEY_PATTERN.search(line.split()[0]) |
| 488 | temp[name_search.group(1)] = line |
Chris Gross | fabf50a | 2019-05-02 12:42:09 -0700 | [diff] [blame] | 489 | return temp |
| 490 | |
Daniel Norman | 0374741 | 2022-02-25 10:38:37 -0800 | [diff] [blame^] | 491 | framework_dict = read_helper(framework_meta_dir) |
| 492 | vendor_dict = read_helper(vendor_meta_dir) |
Bill Peckham | 19c3feb | 2020-03-20 18:31:43 -0700 | [diff] [blame] | 493 | merged_dict = {} |
Chris Gross | fabf50a | 2019-05-02 12:42:09 -0700 | [diff] [blame] | 494 | |
Bill Peckham | 19c3feb | 2020-03-20 18:31:43 -0700 | [diff] [blame] | 495 | def filter_into_merged_dict(item_dict, partition_set): |
| 496 | for key, value in item_dict.items(): |
Daniel Norman | 0374741 | 2022-02-25 10:38:37 -0800 | [diff] [blame^] | 497 | tag_search = PARTITION_TAG_PATTERN.search(value) |
Bill Peckham | 19c3feb | 2020-03-20 18:31:43 -0700 | [diff] [blame] | 498 | |
Daniel Norman | 0374741 | 2022-02-25 10:38:37 -0800 | [diff] [blame^] | 499 | if tag_search is None: |
Bill Peckham | 19c3feb | 2020-03-20 18:31:43 -0700 | [diff] [blame] | 500 | raise ValueError('Entry missing partition tag: %s' % value) |
| 501 | |
Daniel Norman | 0374741 | 2022-02-25 10:38:37 -0800 | [diff] [blame^] | 502 | partition_tag = tag_search.group(1) |
Bill Peckham | 19c3feb | 2020-03-20 18:31:43 -0700 | [diff] [blame] | 503 | |
| 504 | if partition_tag in partition_set: |
| 505 | if key in merged_dict: |
Daniel Norman | b0c7591 | 2020-09-24 14:30:21 -0700 | [diff] [blame] | 506 | if OPTIONS.allow_duplicate_apkapex_keys: |
| 507 | # TODO(b/150582573) Always raise on duplicates. |
| 508 | logger.warning('Duplicate key %s' % key) |
| 509 | continue |
| 510 | else: |
| 511 | raise ValueError('Duplicate key %s' % key) |
Bill Peckham | 19c3feb | 2020-03-20 18:31:43 -0700 | [diff] [blame] | 512 | |
| 513 | merged_dict[key] = value |
| 514 | |
Daniel Norman | 0374741 | 2022-02-25 10:38:37 -0800 | [diff] [blame^] | 515 | # Prioritize framework keys first. |
| 516 | # Duplicate keys from vendor are an error, or ignored. |
| 517 | filter_into_merged_dict(framework_dict, OPTIONS.framework_partition_set) |
| 518 | filter_into_merged_dict(vendor_dict, OPTIONS.vendor_partition_set) |
Chris Gross | fabf50a | 2019-05-02 12:42:09 -0700 | [diff] [blame] | 519 | |
Bill Peckham | 19c3feb | 2020-03-20 18:31:43 -0700 | [diff] [blame] | 520 | # The following code is similar to write_sorted_data, but different enough |
| 521 | # that we couldn't use that function. We need the output to be sorted by the |
| 522 | # basename of the apex/apk (without the ".apex" or ".apk" suffix). This |
| 523 | # allows the sort to be consistent with the framework/vendor input data and |
| 524 | # eases comparison of input data with merged data. |
Daniel Norman | 0374741 | 2022-02-25 10:38:37 -0800 | [diff] [blame^] | 525 | with open(os.path.join(merged_meta_dir, file_name), 'w') as output: |
| 526 | for key, value in sorted(merged_dict.items()): |
| 527 | output.write(value + '\n') |
Daniel Norman | a61cde0 | 2019-05-03 14:19:13 -0700 | [diff] [blame] | 528 | |
| 529 | |
Daniel Norman | 0374741 | 2022-02-25 10:38:37 -0800 | [diff] [blame^] | 530 | def create_file_contexts_copies(framework_meta_dir, vendor_meta_dir, |
| 531 | merged_meta_dir): |
| 532 | """Creates named copies of each partial build's file_contexts.bin. |
Daniel Norman | d5d70ea | 2019-06-05 15:13:43 -0700 | [diff] [blame] | 533 | |
Daniel Norman | 0374741 | 2022-02-25 10:38:37 -0800 | [diff] [blame^] | 534 | Used when regenerating images from the partial build. |
| 535 | """ |
| 536 | |
| 537 | def copy_fc_file(source_dir, file_name): |
| 538 | for name in (file_name, 'file_contexts.bin'): |
| 539 | fc_path = os.path.join(source_dir, name) |
| 540 | if os.path.exists(fc_path): |
| 541 | shutil.copyfile(fc_path, os.path.join(merged_meta_dir, file_name)) |
| 542 | return |
| 543 | raise ValueError('Missing file_contexts file from %s: %s', source_dir, |
| 544 | file_name) |
| 545 | |
| 546 | copy_fc_file(framework_meta_dir, 'framework_file_contexts.bin') |
| 547 | copy_fc_file(vendor_meta_dir, 'vendor_file_contexts.bin') |
| 548 | |
| 549 | # Replace <image>_selinux_fc values with framework or vendor file_contexts.bin |
| 550 | # depending on which dictionary the key came from. |
| 551 | # Only the file basename is required because all selinux_fc properties are |
| 552 | # replaced with the full path to the file under META/ when misc_info.txt is |
| 553 | # loaded from target files for repacking. See common.py LoadInfoDict(). |
| 554 | for key in OPTIONS.vendor_misc_info: |
| 555 | if key.endswith('_selinux_fc'): |
| 556 | OPTIONS.merged_misc_info[key] = 'vendor_file_contexts.bin' |
| 557 | for key in OPTIONS.framework_misc_info: |
| 558 | if key.endswith('_selinux_fc'): |
| 559 | OPTIONS.merged_misc_info[key] = 'framework_file_contexts.bin' |
Daniel Norman | 72c626f | 2019-05-13 15:58:14 -0700 | [diff] [blame] | 560 | |
| 561 | |
Daniel Norman | 0374741 | 2022-02-25 10:38:37 -0800 | [diff] [blame^] | 562 | def compile_split_sepolicy(target_files_dir, partition_map): |
Daniel Norman | 48603ff | 2021-02-22 15:15:24 -0800 | [diff] [blame] | 563 | """Uses secilc to compile a split sepolicy file. |
| 564 | |
| 565 | Depends on various */etc/selinux/* and */etc/vintf/* files within partitions. |
| 566 | |
| 567 | Args: |
Daniel Norman | 0374741 | 2022-02-25 10:38:37 -0800 | [diff] [blame^] | 568 | target_files_dir: Extracted directory of target_files, containing partition |
| 569 | directories. |
| 570 | partition_map: A map of partition name -> relative path within |
| 571 | target_files_dir. |
Daniel Norman | 48603ff | 2021-02-22 15:15:24 -0800 | [diff] [blame] | 572 | |
| 573 | Returns: |
| 574 | A command list that can be executed to create the compiled sepolicy. |
| 575 | """ |
| 576 | |
| 577 | def get_file(partition, path): |
| 578 | if partition not in partition_map: |
| 579 | logger.warning('Cannot load SEPolicy files for missing partition %s', |
| 580 | partition) |
| 581 | return None |
Daniel Norman | 0374741 | 2022-02-25 10:38:37 -0800 | [diff] [blame^] | 582 | return os.path.join(target_files_dir, partition_map[partition], path) |
Daniel Norman | 48603ff | 2021-02-22 15:15:24 -0800 | [diff] [blame] | 583 | |
| 584 | # Load the kernel sepolicy version from the FCM. This is normally provided |
| 585 | # directly to selinux.cpp as a build flag, but is also available in this file. |
| 586 | fcm_file = get_file('system', 'etc/vintf/compatibility_matrix.device.xml') |
| 587 | if not fcm_file or not os.path.exists(fcm_file): |
| 588 | raise ExternalError('Missing required file for loading sepolicy: %s', fcm) |
| 589 | kernel_sepolicy_version = ElementTree.parse(fcm_file).getroot().find( |
| 590 | 'sepolicy/kernel-sepolicy-version').text |
| 591 | |
| 592 | # Load the vendor's plat sepolicy version. This is the version used for |
| 593 | # locating sepolicy mapping files. |
| 594 | vendor_plat_version_file = get_file('vendor', |
| 595 | 'etc/selinux/plat_sepolicy_vers.txt') |
| 596 | if not vendor_plat_version_file or not os.path.exists( |
Daniel Norman | 2d7989a | 2021-04-05 17:40:47 +0000 | [diff] [blame] | 597 | vendor_plat_version_file): |
Daniel Norman | 48603ff | 2021-02-22 15:15:24 -0800 | [diff] [blame] | 598 | raise ExternalError('Missing required sepolicy file %s', |
| 599 | vendor_plat_version_file) |
| 600 | with open(vendor_plat_version_file) as f: |
| 601 | vendor_plat_version = f.read().strip() |
| 602 | |
| 603 | # Use the same flags and arguments as selinux.cpp OpenSplitPolicy(). |
| 604 | cmd = ['secilc', '-m', '-M', 'true', '-G', '-N'] |
| 605 | cmd.extend(['-c', kernel_sepolicy_version]) |
Daniel Norman | 0374741 | 2022-02-25 10:38:37 -0800 | [diff] [blame^] | 606 | cmd.extend(['-o', os.path.join(target_files_dir, 'META/combined_sepolicy')]) |
Daniel Norman | 48603ff | 2021-02-22 15:15:24 -0800 | [diff] [blame] | 607 | cmd.extend(['-f', '/dev/null']) |
| 608 | |
| 609 | required_policy_files = ( |
| 610 | ('system', 'etc/selinux/plat_sepolicy.cil'), |
| 611 | ('system', 'etc/selinux/mapping/%s.cil' % vendor_plat_version), |
| 612 | ('vendor', 'etc/selinux/vendor_sepolicy.cil'), |
| 613 | ('vendor', 'etc/selinux/plat_pub_versioned.cil'), |
| 614 | ) |
| 615 | for policy in (map(lambda partition_and_path: get_file(*partition_and_path), |
| 616 | required_policy_files)): |
| 617 | if not policy or not os.path.exists(policy): |
| 618 | raise ExternalError('Missing required sepolicy file %s', policy) |
| 619 | cmd.append(policy) |
| 620 | |
| 621 | optional_policy_files = ( |
| 622 | ('system', 'etc/selinux/mapping/%s.compat.cil' % vendor_plat_version), |
| 623 | ('system_ext', 'etc/selinux/system_ext_sepolicy.cil'), |
| 624 | ('system_ext', 'etc/selinux/mapping/%s.cil' % vendor_plat_version), |
| 625 | ('product', 'etc/selinux/product_sepolicy.cil'), |
| 626 | ('product', 'etc/selinux/mapping/%s.cil' % vendor_plat_version), |
| 627 | ('odm', 'etc/selinux/odm_sepolicy.cil'), |
| 628 | ) |
| 629 | for policy in (map(lambda partition_and_path: get_file(*partition_and_path), |
| 630 | optional_policy_files)): |
| 631 | if policy and os.path.exists(policy): |
| 632 | cmd.append(policy) |
| 633 | |
| 634 | return cmd |
| 635 | |
| 636 | |
Daniel Norman | 0374741 | 2022-02-25 10:38:37 -0800 | [diff] [blame^] | 637 | def validate_merged_apex_info(target_files_dir, partitions): |
Daniel Norman | e9af70a | 2021-04-15 16:39:22 -0700 | [diff] [blame] | 638 | """Validates the APEX files in the merged target files directory. |
| 639 | |
| 640 | Checks the APEX files in all possible preinstalled APEX directories. |
| 641 | Depends on the <partition>/apex/* APEX files within partitions. |
| 642 | |
| 643 | Args: |
Daniel Norman | 0374741 | 2022-02-25 10:38:37 -0800 | [diff] [blame^] | 644 | target_files_dir: Extracted directory of target_files, containing partition |
Daniel Norman | 571e182 | 2021-06-25 17:18:25 -0700 | [diff] [blame] | 645 | directories. |
Daniel Norman | e9af70a | 2021-04-15 16:39:22 -0700 | [diff] [blame] | 646 | partitions: A list of all the partitions in the output directory. |
| 647 | |
| 648 | Raises: |
| 649 | RuntimeError: if apex_utils fails to parse any APEX file. |
| 650 | ExternalError: if the same APEX package is provided by multiple partitions. |
| 651 | """ |
| 652 | apex_packages = set() |
| 653 | |
Daniel Norman | 0374741 | 2022-02-25 10:38:37 -0800 | [diff] [blame^] | 654 | apex_partitions = ('system', 'system_ext', 'product', 'vendor', 'odm') |
Daniel Norman | e9af70a | 2021-04-15 16:39:22 -0700 | [diff] [blame] | 655 | for partition in filter(lambda p: p in apex_partitions, partitions): |
| 656 | apex_info = apex_utils.GetApexInfoFromTargetFiles( |
Daniel Norman | 0374741 | 2022-02-25 10:38:37 -0800 | [diff] [blame^] | 657 | target_files_dir, partition, compressed_only=False) |
Daniel Norman | e9af70a | 2021-04-15 16:39:22 -0700 | [diff] [blame] | 658 | partition_apex_packages = set([info.package_name for info in apex_info]) |
| 659 | duplicates = apex_packages.intersection(partition_apex_packages) |
| 660 | if duplicates: |
| 661 | raise ExternalError( |
| 662 | 'Duplicate APEX packages found in multiple partitions: %s' % |
| 663 | ' '.join(duplicates)) |
| 664 | apex_packages.update(partition_apex_packages) |
| 665 | |
| 666 | |
Daniel Norman | 0374741 | 2022-02-25 10:38:37 -0800 | [diff] [blame^] | 667 | def generate_care_map(partitions, target_files_dir): |
| 668 | """Generates a merged META/care_map.pb file in the target files dir. |
Daniel Norman | db8cacc | 2021-04-09 15:34:43 -0700 | [diff] [blame] | 669 | |
| 670 | Depends on the info dict from META/misc_info.txt, as well as built images |
| 671 | within IMAGES/. |
| 672 | |
| 673 | Args: |
| 674 | partitions: A list of partitions to potentially include in the care map. |
Daniel Norman | 0374741 | 2022-02-25 10:38:37 -0800 | [diff] [blame^] | 675 | target_files_dir: Extracted directory of target_files, containing partition |
| 676 | directories. |
Daniel Norman | db8cacc | 2021-04-09 15:34:43 -0700 | [diff] [blame] | 677 | """ |
Daniel Norman | 0374741 | 2022-02-25 10:38:37 -0800 | [diff] [blame^] | 678 | OPTIONS.info_dict = common.LoadInfoDict(target_files_dir) |
Daniel Norman | db8cacc | 2021-04-09 15:34:43 -0700 | [diff] [blame] | 679 | partition_image_map = {} |
| 680 | for partition in partitions: |
Daniel Norman | 0374741 | 2022-02-25 10:38:37 -0800 | [diff] [blame^] | 681 | image_path = os.path.join(target_files_dir, 'IMAGES', |
Daniel Norman | db8cacc | 2021-04-09 15:34:43 -0700 | [diff] [blame] | 682 | '{}.img'.format(partition)) |
| 683 | if os.path.exists(image_path): |
| 684 | partition_image_map[partition] = image_path |
| 685 | # Regenerated images should have their image_size property already set. |
| 686 | image_size_prop = '{}_image_size'.format(partition) |
| 687 | if image_size_prop not in OPTIONS.info_dict: |
| 688 | # Images copied directly from input target files packages will need |
| 689 | # their image sizes calculated. |
| 690 | partition_size = sparse_img.GetImagePartitionSize(image_path) |
| 691 | image_props = build_image.ImagePropFromGlobalDict( |
| 692 | OPTIONS.info_dict, partition) |
| 693 | verity_image_builder = verity_utils.CreateVerityImageBuilder( |
| 694 | image_props) |
| 695 | image_size = verity_image_builder.CalculateMaxImageSize(partition_size) |
| 696 | OPTIONS.info_dict[image_size_prop] = image_size |
| 697 | |
Daniel Norman | db8cacc | 2021-04-09 15:34:43 -0700 | [diff] [blame] | 698 | |
Daniel Norman | 0374741 | 2022-02-25 10:38:37 -0800 | [diff] [blame^] | 699 | def merge_meta_files(temp_dir, merged_dir): |
| 700 | """Merges various files in META/*.""" |
Bill Peckham | e9eb5f9 | 2019-02-01 15:52:10 -0800 | [diff] [blame] | 701 | |
Daniel Norman | 0374741 | 2022-02-25 10:38:37 -0800 | [diff] [blame^] | 702 | framework_meta_dir = os.path.join(temp_dir, 'framework_meta', 'META') |
| 703 | extract_items( |
| 704 | input_zip=OPTIONS.framework_target_files, |
| 705 | output_dir=os.path.dirname(framework_meta_dir), |
| 706 | extract_item_list=('META/*',)) |
Bill Peckham | e9eb5f9 | 2019-02-01 15:52:10 -0800 | [diff] [blame] | 707 | |
Daniel Norman | 0374741 | 2022-02-25 10:38:37 -0800 | [diff] [blame^] | 708 | vendor_meta_dir = os.path.join(temp_dir, 'vendor_meta', 'META') |
| 709 | extract_items( |
| 710 | input_zip=OPTIONS.vendor_target_files, |
| 711 | output_dir=os.path.dirname(vendor_meta_dir), |
| 712 | extract_item_list=('META/*',)) |
Bill Peckham | e9eb5f9 | 2019-02-01 15:52:10 -0800 | [diff] [blame] | 713 | |
Daniel Norman | 0374741 | 2022-02-25 10:38:37 -0800 | [diff] [blame^] | 714 | merged_meta_dir = os.path.join(merged_dir, 'META') |
Bill Peckham | 364c1cc | 2019-03-29 18:27:23 -0700 | [diff] [blame] | 715 | |
Daniel Norman | 0374741 | 2022-02-25 10:38:37 -0800 | [diff] [blame^] | 716 | # Merge META/misc_info.txt into OPTIONS.merged_misc_info, |
| 717 | # but do not write it yet. The following functions may further |
| 718 | # modify this dict. |
| 719 | OPTIONS.merged_misc_info = merge_misc_info_txt( |
| 720 | framework_meta_dir=framework_meta_dir, |
| 721 | vendor_meta_dir=vendor_meta_dir, |
| 722 | merged_meta_dir=merged_meta_dir) |
Daniel Norman | 72c626f | 2019-05-13 15:58:14 -0700 | [diff] [blame] | 723 | |
Daniel Norman | 0374741 | 2022-02-25 10:38:37 -0800 | [diff] [blame^] | 724 | create_file_contexts_copies( |
| 725 | framework_meta_dir=framework_meta_dir, |
| 726 | vendor_meta_dir=vendor_meta_dir, |
| 727 | merged_meta_dir=merged_meta_dir) |
Bill Peckham | e9eb5f9 | 2019-02-01 15:52:10 -0800 | [diff] [blame] | 728 | |
Daniel Norman | 0374741 | 2022-02-25 10:38:37 -0800 | [diff] [blame^] | 729 | if OPTIONS.merged_misc_info['use_dynamic_partitions'] == 'true': |
| 730 | merge_dynamic_partitions_info_txt( |
| 731 | framework_meta_dir=framework_meta_dir, |
| 732 | vendor_meta_dir=vendor_meta_dir, |
| 733 | merged_meta_dir=merged_meta_dir) |
Daniel Norman | a61cde0 | 2019-05-03 14:19:13 -0700 | [diff] [blame] | 734 | |
Daniel Norman | 0374741 | 2022-02-25 10:38:37 -0800 | [diff] [blame^] | 735 | if OPTIONS.merged_misc_info['ab_update'] == 'true': |
| 736 | merge_ab_partitions_txt( |
| 737 | framework_meta_dir=framework_meta_dir, |
| 738 | vendor_meta_dir=vendor_meta_dir, |
| 739 | merged_meta_dir=merged_meta_dir) |
Chris Gross | fabf50a | 2019-05-02 12:42:09 -0700 | [diff] [blame] | 740 | |
Daniel Norman | 0374741 | 2022-02-25 10:38:37 -0800 | [diff] [blame^] | 741 | for file_name in ('apkcerts.txt', 'apexkeys.txt'): |
| 742 | merge_package_keys_txt( |
| 743 | framework_meta_dir=framework_meta_dir, |
| 744 | vendor_meta_dir=vendor_meta_dir, |
| 745 | merged_meta_dir=merged_meta_dir, |
| 746 | file_name=file_name) |
Chris Gross | fabf50a | 2019-05-02 12:42:09 -0700 | [diff] [blame] | 747 | |
Daniel Norman | 0374741 | 2022-02-25 10:38:37 -0800 | [diff] [blame^] | 748 | # Write the now-finalized OPTIONS.merged_misc_info. |
| 749 | write_sorted_data( |
| 750 | data=OPTIONS.merged_misc_info, |
| 751 | path=os.path.join(merged_meta_dir, 'misc_info.txt')) |
Jose Galmes | 9c8f6eb | 2021-07-21 09:34:08 -0700 | [diff] [blame] | 752 | |
| 753 | |
Daniel Norman | 0374741 | 2022-02-25 10:38:37 -0800 | [diff] [blame^] | 754 | def process_dexopt(temp_dir, output_target_files_dir): |
Jose Galmes | 9c8f6eb | 2021-07-21 09:34:08 -0700 | [diff] [blame] | 755 | """If needed, generates dexopt files for vendor apps. |
| 756 | |
| 757 | Args: |
| 758 | temp_dir: Location containing an 'output' directory where target files have |
Daniel Norman | e292f5f | 2022-02-17 14:16:10 -0800 | [diff] [blame] | 759 | been extracted, e.g. <temp_dir>/output/SYSTEM, <temp_dir>/output/IMAGES, |
| 760 | etc. |
Daniel Norman | 0374741 | 2022-02-25 10:38:37 -0800 | [diff] [blame^] | 761 | output_target_files_dir: The name of a directory that will be used to create |
| 762 | the output target files package after all the special cases are processed. |
Jose Galmes | 9c8f6eb | 2021-07-21 09:34:08 -0700 | [diff] [blame] | 763 | """ |
| 764 | # Load vendor and framework META/misc_info.txt. |
Daniel Norman | 0374741 | 2022-02-25 10:38:37 -0800 | [diff] [blame^] | 765 | if (OPTIONS.vendor_misc_info.get('building_with_vsdk') != 'true' or |
| 766 | OPTIONS.framework_dexpreopt_tools is None or |
| 767 | OPTIONS.framework_dexpreopt_config is None or |
| 768 | OPTIONS.vendor_dexpreopt_config is None): |
Jose Galmes | 9c8f6eb | 2021-07-21 09:34:08 -0700 | [diff] [blame] | 769 | return |
| 770 | |
| 771 | logger.info('applying dexpreopt') |
| 772 | |
| 773 | # The directory structure to apply dexpreopt is: |
| 774 | # |
| 775 | # <temp_dir>/ |
| 776 | # framework_meta/ |
| 777 | # META/ |
| 778 | # vendor_meta/ |
| 779 | # META/ |
| 780 | # output/ |
| 781 | # SYSTEM/ |
| 782 | # VENDOR/ |
| 783 | # IMAGES/ |
| 784 | # <other items extracted from system and vendor target files> |
| 785 | # tools/ |
| 786 | # <contents of dexpreopt_tools.zip> |
| 787 | # system_config/ |
| 788 | # <contents of system dexpreopt_config.zip> |
| 789 | # vendor_config/ |
| 790 | # <contents of vendor dexpreopt_config.zip> |
| 791 | # system -> output/SYSTEM |
| 792 | # vendor -> output/VENDOR |
| 793 | # apex -> output/SYSTEM/apex (only for flattened APEX builds) |
| 794 | # apex/ (extracted updatable APEX) |
| 795 | # <apex 1>/ |
| 796 | # ... |
| 797 | # <apex 2>/ |
| 798 | # ... |
| 799 | # ... |
| 800 | # out/dex2oat_result/vendor/ |
| 801 | # <app> |
| 802 | # oat/arm64/ |
| 803 | # package.vdex |
| 804 | # package.odex |
| 805 | # <priv-app> |
| 806 | # oat/arm64/ |
| 807 | # package.vdex |
| 808 | # package.odex |
| 809 | dexpreopt_tools_files_temp_dir = os.path.join(temp_dir, 'tools') |
Daniel Norman | e292f5f | 2022-02-17 14:16:10 -0800 | [diff] [blame] | 810 | dexpreopt_framework_config_files_temp_dir = os.path.join( |
| 811 | temp_dir, 'system_config') |
| 812 | dexpreopt_vendor_config_files_temp_dir = os.path.join(temp_dir, |
| 813 | 'vendor_config') |
Jose Galmes | 9c8f6eb | 2021-07-21 09:34:08 -0700 | [diff] [blame] | 814 | |
| 815 | extract_items( |
Daniel Norman | 0374741 | 2022-02-25 10:38:37 -0800 | [diff] [blame^] | 816 | input_zip=OPTIONS.framework_dexpreopt_tools, |
| 817 | output_dir=dexpreopt_tools_files_temp_dir, |
Jose Galmes | 9c8f6eb | 2021-07-21 09:34:08 -0700 | [diff] [blame] | 818 | extract_item_list=('*',)) |
| 819 | extract_items( |
Daniel Norman | 0374741 | 2022-02-25 10:38:37 -0800 | [diff] [blame^] | 820 | input_zip=OPTIONS.framework_dexpreopt_config, |
| 821 | output_dir=dexpreopt_framework_config_files_temp_dir, |
Jose Galmes | 9c8f6eb | 2021-07-21 09:34:08 -0700 | [diff] [blame] | 822 | extract_item_list=('*',)) |
| 823 | extract_items( |
Daniel Norman | 0374741 | 2022-02-25 10:38:37 -0800 | [diff] [blame^] | 824 | input_zip=OPTIONS.vendor_dexpreopt_config, |
| 825 | output_dir=dexpreopt_vendor_config_files_temp_dir, |
Jose Galmes | 9c8f6eb | 2021-07-21 09:34:08 -0700 | [diff] [blame] | 826 | extract_item_list=('*',)) |
| 827 | |
Daniel Norman | e292f5f | 2022-02-17 14:16:10 -0800 | [diff] [blame] | 828 | os.symlink( |
Daniel Norman | 0374741 | 2022-02-25 10:38:37 -0800 | [diff] [blame^] | 829 | os.path.join(output_target_files_dir, 'SYSTEM'), |
Daniel Norman | e292f5f | 2022-02-17 14:16:10 -0800 | [diff] [blame] | 830 | os.path.join(temp_dir, 'system')) |
| 831 | os.symlink( |
Daniel Norman | 0374741 | 2022-02-25 10:38:37 -0800 | [diff] [blame^] | 832 | os.path.join(output_target_files_dir, 'VENDOR'), |
Daniel Norman | e292f5f | 2022-02-17 14:16:10 -0800 | [diff] [blame] | 833 | os.path.join(temp_dir, 'vendor')) |
Jose Galmes | 9c8f6eb | 2021-07-21 09:34:08 -0700 | [diff] [blame] | 834 | |
| 835 | # The directory structure for flatteded APEXes is: |
| 836 | # |
| 837 | # SYSTEM |
| 838 | # apex |
| 839 | # <APEX name, e.g., com.android.wifi> |
| 840 | # apex_manifest.pb |
| 841 | # apex_pubkey |
| 842 | # etc/ |
| 843 | # javalib/ |
| 844 | # lib/ |
| 845 | # lib64/ |
| 846 | # priv-app/ |
| 847 | # |
| 848 | # The directory structure for updatable APEXes is: |
| 849 | # |
| 850 | # SYSTEM |
| 851 | # apex |
| 852 | # com.android.adbd.apex |
| 853 | # com.android.appsearch.apex |
| 854 | # com.android.art.apex |
| 855 | # ... |
Daniel Norman | 0374741 | 2022-02-25 10:38:37 -0800 | [diff] [blame^] | 856 | apex_root = os.path.join(output_target_files_dir, 'SYSTEM', 'apex') |
Jose Galmes | 9c8f6eb | 2021-07-21 09:34:08 -0700 | [diff] [blame] | 857 | |
| 858 | # Check for flattended versus updatable APEX. |
Daniel Norman | 0374741 | 2022-02-25 10:38:37 -0800 | [diff] [blame^] | 859 | if OPTIONS.framework_misc_info.get('target_flatten_apex') == 'false': |
Jose Galmes | 9c8f6eb | 2021-07-21 09:34:08 -0700 | [diff] [blame] | 860 | # Extract APEX. |
| 861 | logging.info('extracting APEX') |
| 862 | |
| 863 | apex_extract_root_dir = os.path.join(temp_dir, 'apex') |
| 864 | os.makedirs(apex_extract_root_dir) |
| 865 | |
| 866 | for apex in (glob.glob(os.path.join(apex_root, '*.apex')) + |
| 867 | glob.glob(os.path.join(apex_root, '*.capex'))): |
| 868 | logging.info(' apex: %s', apex) |
| 869 | # deapexer is in the same directory as the merge_target_files binary extracted |
| 870 | # from otatools.zip. |
| 871 | apex_json_info = subprocess.check_output(['deapexer', 'info', apex]) |
| 872 | logging.info(' info: %s', apex_json_info) |
| 873 | apex_info = json.loads(apex_json_info) |
| 874 | apex_name = apex_info['name'] |
| 875 | logging.info(' name: %s', apex_name) |
| 876 | |
| 877 | apex_extract_dir = os.path.join(apex_extract_root_dir, apex_name) |
| 878 | os.makedirs(apex_extract_dir) |
| 879 | |
| 880 | # deapexer uses debugfs_static, which is part of otatools.zip. |
| 881 | command = [ |
| 882 | 'deapexer', |
| 883 | '--debugfs_path', |
| 884 | 'debugfs_static', |
| 885 | 'extract', |
| 886 | apex, |
| 887 | apex_extract_dir, |
| 888 | ] |
| 889 | logging.info(' running %s', command) |
| 890 | subprocess.check_call(command) |
| 891 | else: |
| 892 | # Flattened APEXes don't need to be extracted since they have the necessary |
| 893 | # directory structure. |
| 894 | os.symlink(os.path.join(apex_root), os.path.join(temp_dir, 'apex')) |
| 895 | |
| 896 | # Modify system config to point to the tools that have been extracted. |
| 897 | # Absolute or .. paths are not allowed by the dexpreopt_gen tool in |
| 898 | # dexpreopt_soong.config. |
| 899 | dexpreopt_framework_soon_config = os.path.join( |
| 900 | dexpreopt_framework_config_files_temp_dir, 'dexpreopt_soong.config') |
| 901 | with open(dexpreopt_framework_soon_config, 'w') as f: |
| 902 | dexpreopt_soong_config = { |
| 903 | 'Profman': 'tools/profman', |
| 904 | 'Dex2oat': 'tools/dex2oatd', |
| 905 | 'Aapt': 'tools/aapt2', |
| 906 | 'SoongZip': 'tools/soong_zip', |
| 907 | 'Zip2zip': 'tools/zip2zip', |
| 908 | 'ManifestCheck': 'tools/manifest_check', |
| 909 | 'ConstructContext': 'tools/construct_context', |
| 910 | } |
| 911 | json.dump(dexpreopt_soong_config, f) |
| 912 | |
| 913 | # TODO(b/188179859): Make *dex location configurable to vendor or system_other. |
| 914 | use_system_other_odex = False |
| 915 | |
| 916 | if use_system_other_odex: |
| 917 | dex_img = 'SYSTEM_OTHER' |
| 918 | else: |
| 919 | dex_img = 'VENDOR' |
| 920 | # Open vendor_filesystem_config to append the items generated by dexopt. |
| 921 | vendor_file_system_config = open( |
Daniel Norman | e292f5f | 2022-02-17 14:16:10 -0800 | [diff] [blame] | 922 | os.path.join(temp_dir, 'output', 'META', |
| 923 | 'vendor_filesystem_config.txt'), 'a') |
Jose Galmes | 9c8f6eb | 2021-07-21 09:34:08 -0700 | [diff] [blame] | 924 | |
| 925 | # Dexpreopt vendor apps. |
| 926 | dexpreopt_config_suffix = '_dexpreopt.config' |
Daniel Norman | e292f5f | 2022-02-17 14:16:10 -0800 | [diff] [blame] | 927 | for config in glob.glob( |
| 928 | os.path.join(dexpreopt_vendor_config_files_temp_dir, |
| 929 | '*' + dexpreopt_config_suffix)): |
Jose Galmes | 9c8f6eb | 2021-07-21 09:34:08 -0700 | [diff] [blame] | 930 | app = os.path.basename(config)[:-len(dexpreopt_config_suffix)] |
| 931 | logging.info('dexpreopt config: %s %s', config, app) |
| 932 | |
| 933 | apk_dir = 'app' |
| 934 | apk_path = os.path.join(temp_dir, 'vendor', apk_dir, app, app + '.apk') |
| 935 | if not os.path.exists(apk_path): |
| 936 | apk_dir = 'priv-app' |
| 937 | apk_path = os.path.join(temp_dir, 'vendor', apk_dir, app, app + '.apk') |
| 938 | if not os.path.exists(apk_path): |
Daniel Norman | e292f5f | 2022-02-17 14:16:10 -0800 | [diff] [blame] | 939 | logging.warning( |
| 940 | 'skipping dexpreopt for %s, no apk found in vendor/app ' |
| 941 | 'or vendor/priv-app', app) |
Jose Galmes | 9c8f6eb | 2021-07-21 09:34:08 -0700 | [diff] [blame] | 942 | continue |
| 943 | |
| 944 | # Generate dexpreopting script. Note 'out_dir' is not the output directory |
| 945 | # where the script is generated, but the OUT_DIR at build time referenced |
| 946 | # in the dexpreot config files, e.g., "out/.../core-oj.jar", so the tool knows |
| 947 | # how to adjust the path. |
| 948 | command = [ |
| 949 | os.path.join(dexpreopt_tools_files_temp_dir, 'dexpreopt_gen'), |
| 950 | '-global', |
Daniel Norman | e292f5f | 2022-02-17 14:16:10 -0800 | [diff] [blame] | 951 | os.path.join(dexpreopt_framework_config_files_temp_dir, |
| 952 | 'dexpreopt.config'), |
Jose Galmes | 9c8f6eb | 2021-07-21 09:34:08 -0700 | [diff] [blame] | 953 | '-global_soong', |
Daniel Norman | e292f5f | 2022-02-17 14:16:10 -0800 | [diff] [blame] | 954 | os.path.join(dexpreopt_framework_config_files_temp_dir, |
| 955 | 'dexpreopt_soong.config'), |
Jose Galmes | 9c8f6eb | 2021-07-21 09:34:08 -0700 | [diff] [blame] | 956 | '-module', |
| 957 | config, |
| 958 | '-dexpreopt_script', |
| 959 | 'dexpreopt_app.sh', |
| 960 | '-out_dir', |
| 961 | 'out', |
| 962 | '-base_path', |
| 963 | '.', |
| 964 | '--uses_target_files', |
| 965 | ] |
| 966 | |
| 967 | # Run the command from temp_dir so all tool paths are its descendants. |
Daniel Norman | e292f5f | 2022-02-17 14:16:10 -0800 | [diff] [blame] | 968 | logging.info('running %s', command) |
| 969 | subprocess.check_call(command, cwd=temp_dir) |
Jose Galmes | 9c8f6eb | 2021-07-21 09:34:08 -0700 | [diff] [blame] | 970 | |
| 971 | # Call the generated script. |
| 972 | command = ['sh', 'dexpreopt_app.sh', apk_path] |
Daniel Norman | e292f5f | 2022-02-17 14:16:10 -0800 | [diff] [blame] | 973 | logging.info('running %s', command) |
| 974 | subprocess.check_call(command, cwd=temp_dir) |
Jose Galmes | 9c8f6eb | 2021-07-21 09:34:08 -0700 | [diff] [blame] | 975 | |
| 976 | # Output files are in: |
| 977 | # |
| 978 | # <temp_dir>/out/dex2oat_result/vendor/priv-app/<app>/oat/arm64/package.vdex |
| 979 | # <temp_dir>/out/dex2oat_result/vendor/priv-app/<app>/oat/arm64/package.odex |
| 980 | # <temp_dir>/out/dex2oat_result/vendor/app/<app>/oat/arm64/package.vdex |
| 981 | # <temp_dir>/out/dex2oat_result/vendor/app/<app>/oat/arm64/package.odex |
| 982 | # |
| 983 | # Copy the files to their destination. The structure of system_other is: |
| 984 | # |
| 985 | # system_other/ |
| 986 | # system-other-odex-marker |
| 987 | # system/ |
| 988 | # app/ |
| 989 | # <app>/oat/arm64/ |
| 990 | # <app>.odex |
| 991 | # <app>.vdex |
| 992 | # ... |
| 993 | # priv-app/ |
| 994 | # <app>/oat/arm64/ |
| 995 | # <app>.odex |
| 996 | # <app>.vdex |
| 997 | # ... |
| 998 | |
| 999 | # TODO(b/188179859): Support for other architectures. |
| 1000 | arch = 'arm64' |
| 1001 | |
Daniel Norman | e292f5f | 2022-02-17 14:16:10 -0800 | [diff] [blame] | 1002 | dex_destination = os.path.join(temp_dir, 'output', dex_img, apk_dir, app, |
| 1003 | 'oat', arch) |
Jose Galmes | 9c8f6eb | 2021-07-21 09:34:08 -0700 | [diff] [blame] | 1004 | os.makedirs(dex_destination) |
Daniel Norman | e292f5f | 2022-02-17 14:16:10 -0800 | [diff] [blame] | 1005 | dex2oat_path = os.path.join(temp_dir, 'out', 'dex2oat_result', 'vendor', |
| 1006 | apk_dir, app, 'oat', arch) |
| 1007 | shutil.copy( |
| 1008 | os.path.join(dex2oat_path, 'package.vdex'), |
| 1009 | os.path.join(dex_destination, app + '.vdex')) |
| 1010 | shutil.copy( |
| 1011 | os.path.join(dex2oat_path, 'package.odex'), |
| 1012 | os.path.join(dex_destination, app + '.odex')) |
Jose Galmes | 9c8f6eb | 2021-07-21 09:34:08 -0700 | [diff] [blame] | 1013 | |
| 1014 | # Append entries to vendor_file_system_config.txt, such as: |
| 1015 | # |
| 1016 | # vendor/app/<app>/oat 0 2000 755 selabel=u:object_r:vendor_app_file:s0 capabilities=0x0 |
| 1017 | # vendor/app/<app>/oat/arm64 0 2000 755 selabel=u:object_r:vendor_app_file:s0 capabilities=0x0 |
| 1018 | # vendor/app/<app>/oat/arm64/<app>.odex 0 0 644 selabel=u:object_r:vendor_app_file:s0 capabilities=0x0 |
| 1019 | # vendor/app/<app>/oat/arm64/<app>.vdex 0 0 644 selabel=u:object_r:vendor_app_file:s0 capabilities=0x0 |
| 1020 | if not use_system_other_odex: |
| 1021 | vendor_app_prefix = 'vendor/' + apk_dir + '/' + app + '/oat' |
| 1022 | selabel = 'selabel=u:object_r:vendor_app_file:s0 capabilities=0x0' |
| 1023 | vendor_file_system_config.writelines([ |
| 1024 | vendor_app_prefix + ' 0 2000 755 ' + selabel + '\n', |
| 1025 | vendor_app_prefix + '/' + arch + ' 0 2000 755 ' + selabel + '\n', |
Daniel Norman | e292f5f | 2022-02-17 14:16:10 -0800 | [diff] [blame] | 1026 | vendor_app_prefix + '/' + arch + '/' + app + '.odex 0 0 644 ' + |
| 1027 | selabel + '\n', |
| 1028 | vendor_app_prefix + '/' + arch + '/' + app + '.vdex 0 0 644 ' + |
| 1029 | selabel + '\n', |
Jose Galmes | 9c8f6eb | 2021-07-21 09:34:08 -0700 | [diff] [blame] | 1030 | ]) |
| 1031 | |
| 1032 | if not use_system_other_odex: |
| 1033 | vendor_file_system_config.close() |
| 1034 | # Delete vendor.img so that it will be regenerated. |
| 1035 | # TODO(b/188179859): Rebuilding a vendor image in GRF mode (e.g., T(framework) |
| 1036 | # and S(vendor) may require logic similar to that in |
| 1037 | # rebuild_image_with_sepolicy. |
Daniel Norman | 0374741 | 2022-02-25 10:38:37 -0800 | [diff] [blame^] | 1038 | vendor_img = os.path.join(output_target_files_dir, 'IMAGES', 'vendor.img') |
Jose Galmes | 9c8f6eb | 2021-07-21 09:34:08 -0700 | [diff] [blame] | 1039 | if os.path.exists(vendor_img): |
| 1040 | logging.info('Deleting %s', vendor_img) |
| 1041 | os.remove(vendor_img) |
| 1042 | |
Bill Peckham | e9eb5f9 | 2019-02-01 15:52:10 -0800 | [diff] [blame] | 1043 | |
Daniel Norman | 0374741 | 2022-02-25 10:38:37 -0800 | [diff] [blame^] | 1044 | def create_merged_package(temp_dir): |
Tao Bao | 2ad4b82 | 2019-06-27 16:52:12 -0700 | [diff] [blame] | 1045 | """Merges two target files packages into one target files structure. |
Bill Peckham | e9eb5f9 | 2019-02-01 15:52:10 -0800 | [diff] [blame] | 1046 | |
Kiyoung Kim | 7cbeda7 | 2019-06-28 13:26:04 +0900 | [diff] [blame] | 1047 | Returns: |
| 1048 | Path to merged package under temp directory. |
| 1049 | """ |
Daniel Norman | dbbf5a3 | 2020-10-22 16:03:32 -0700 | [diff] [blame] | 1050 | # Extract "as is" items from the input framework and vendor partial target |
| 1051 | # files packages directly into the output temporary directory, since these items |
| 1052 | # do not need special case processing. |
Daniel Norman | 2c99c5b | 2019-03-07 13:01:48 -0800 | [diff] [blame] | 1053 | |
Bill Peckham | e9eb5f9 | 2019-02-01 15:52:10 -0800 | [diff] [blame] | 1054 | output_target_files_temp_dir = os.path.join(temp_dir, 'output') |
Bill Peckham | 889b0c6 | 2019-02-21 18:53:37 -0800 | [diff] [blame] | 1055 | extract_items( |
Daniel Norman | 0374741 | 2022-02-25 10:38:37 -0800 | [diff] [blame^] | 1056 | input_zip=OPTIONS.framework_target_files, |
| 1057 | output_dir=output_target_files_temp_dir, |
| 1058 | extract_item_list=OPTIONS.framework_item_list) |
Bill Peckham | 889b0c6 | 2019-02-21 18:53:37 -0800 | [diff] [blame] | 1059 | extract_items( |
Daniel Norman | 0374741 | 2022-02-25 10:38:37 -0800 | [diff] [blame^] | 1060 | input_zip=OPTIONS.vendor_target_files, |
| 1061 | output_dir=output_target_files_temp_dir, |
| 1062 | extract_item_list=OPTIONS.vendor_item_list) |
Bill Peckham | e9eb5f9 | 2019-02-01 15:52:10 -0800 | [diff] [blame] | 1063 | |
Daniel Norman | dbbf5a3 | 2020-10-22 16:03:32 -0700 | [diff] [blame] | 1064 | # Perform special case processing on META/* items. |
| 1065 | # After this function completes successfully, all the files we need to create |
| 1066 | # the output target files package are in place. |
Daniel Norman | 0374741 | 2022-02-25 10:38:37 -0800 | [diff] [blame^] | 1067 | merge_meta_files(temp_dir=temp_dir, merged_dir=output_target_files_temp_dir) |
| 1068 | |
| 1069 | process_dexopt( |
| 1070 | temp_dir=temp_dir, output_target_files_dir=output_target_files_temp_dir) |
Bill Peckham | e9eb5f9 | 2019-02-01 15:52:10 -0800 | [diff] [blame] | 1071 | |
Kiyoung Kim | 7cbeda7 | 2019-06-28 13:26:04 +0900 | [diff] [blame] | 1072 | return output_target_files_temp_dir |
| 1073 | |
| 1074 | |
Daniel Norman | 0374741 | 2022-02-25 10:38:37 -0800 | [diff] [blame^] | 1075 | def generate_missing_images(target_files_dir): |
| 1076 | """Generate any missing images from target files.""" |
Kiyoung Kim | 7cbeda7 | 2019-06-28 13:26:04 +0900 | [diff] [blame] | 1077 | |
| 1078 | # Regenerate IMAGES in the target directory. |
Bill Peckham | e9eb5f9 | 2019-02-01 15:52:10 -0800 | [diff] [blame] | 1079 | |
Daniel Norman | dbbf5a3 | 2020-10-22 16:03:32 -0700 | [diff] [blame] | 1080 | add_img_args = [ |
| 1081 | '--verbose', |
| 1082 | '--add_missing', |
| 1083 | ] |
Daniel Norman | 0374741 | 2022-02-25 10:38:37 -0800 | [diff] [blame^] | 1084 | if OPTIONS.rebuild_recovery: |
Daniel Norman | a4911da | 2019-03-15 14:36:21 -0700 | [diff] [blame] | 1085 | add_img_args.append('--rebuild_recovery') |
Kiyoung Kim | 7cbeda7 | 2019-06-28 13:26:04 +0900 | [diff] [blame] | 1086 | add_img_args.append(target_files_dir) |
Bill Peckham | e9eb5f9 | 2019-02-01 15:52:10 -0800 | [diff] [blame] | 1087 | |
| 1088 | add_img_to_target_files.main(add_img_args) |
| 1089 | |
Kiyoung Kim | 7cbeda7 | 2019-06-28 13:26:04 +0900 | [diff] [blame] | 1090 | |
Daniel Norman | 0374741 | 2022-02-25 10:38:37 -0800 | [diff] [blame^] | 1091 | def rebuild_image_with_sepolicy(target_files_dir): |
Daniel Norman | 571e182 | 2021-06-25 17:18:25 -0700 | [diff] [blame] | 1092 | """Rebuilds odm.img or vendor.img to include merged sepolicy files. |
| 1093 | |
| 1094 | If odm is present then odm is preferred -- otherwise vendor is used. |
Daniel Norman | 571e182 | 2021-06-25 17:18:25 -0700 | [diff] [blame] | 1095 | """ |
| 1096 | partition = 'vendor' |
| 1097 | if os.path.exists(os.path.join(target_files_dir, 'ODM')) or os.path.exists( |
| 1098 | os.path.join(target_files_dir, 'IMAGES/odm.img')): |
| 1099 | partition = 'odm' |
| 1100 | partition_img = '{}.img'.format(partition) |
Po Hu | a6c5912 | 2022-02-16 08:41:29 +0000 | [diff] [blame] | 1101 | partition_map = '{}.map'.format(partition) |
Daniel Norman | 571e182 | 2021-06-25 17:18:25 -0700 | [diff] [blame] | 1102 | |
| 1103 | logger.info('Recompiling %s using the merged sepolicy files.', partition_img) |
| 1104 | |
| 1105 | # Copy the combined SEPolicy file and framework hashes to the image that is |
| 1106 | # being rebuilt. |
| 1107 | def copy_selinux_file(input_path, output_filename): |
Po Hu | 0e4403e | 2021-07-06 17:05:56 +0800 | [diff] [blame] | 1108 | input_filename = os.path.join(target_files_dir, input_path) |
| 1109 | if not os.path.exists(input_filename): |
| 1110 | input_filename = input_filename.replace('SYSTEM_EXT/', 'SYSTEM/system_ext/') \ |
| 1111 | .replace('PRODUCT/', 'SYSTEM/product/') |
| 1112 | if not os.path.exists(input_filename): |
| 1113 | logger.info('Skipping copy_selinux_file for %s', input_filename) |
| 1114 | return |
Daniel Norman | 571e182 | 2021-06-25 17:18:25 -0700 | [diff] [blame] | 1115 | shutil.copy( |
Po Hu | 0e4403e | 2021-07-06 17:05:56 +0800 | [diff] [blame] | 1116 | input_filename, |
Daniel Norman | 571e182 | 2021-06-25 17:18:25 -0700 | [diff] [blame] | 1117 | os.path.join(target_files_dir, partition.upper(), 'etc/selinux', |
| 1118 | output_filename)) |
| 1119 | |
| 1120 | copy_selinux_file('META/combined_sepolicy', 'precompiled_sepolicy') |
| 1121 | copy_selinux_file('SYSTEM/etc/selinux/plat_sepolicy_and_mapping.sha256', |
| 1122 | 'precompiled_sepolicy.plat_sepolicy_and_mapping.sha256') |
| 1123 | copy_selinux_file( |
| 1124 | 'SYSTEM_EXT/etc/selinux/system_ext_sepolicy_and_mapping.sha256', |
| 1125 | 'precompiled_sepolicy.system_ext_sepolicy_and_mapping.sha256') |
| 1126 | copy_selinux_file('PRODUCT/etc/selinux/product_sepolicy_and_mapping.sha256', |
| 1127 | 'precompiled_sepolicy.product_sepolicy_and_mapping.sha256') |
| 1128 | |
Daniel Norman | 0374741 | 2022-02-25 10:38:37 -0800 | [diff] [blame^] | 1129 | if not OPTIONS.vendor_otatools: |
Daniel Norman | 571e182 | 2021-06-25 17:18:25 -0700 | [diff] [blame] | 1130 | # Remove the partition from the merged target-files archive. It will be |
Daniel Norman | 0374741 | 2022-02-25 10:38:37 -0800 | [diff] [blame^] | 1131 | # rebuilt later automatically by generate_missing_images(). |
Daniel Norman | 571e182 | 2021-06-25 17:18:25 -0700 | [diff] [blame] | 1132 | os.remove(os.path.join(target_files_dir, 'IMAGES', partition_img)) |
Daniel Norman | 0374741 | 2022-02-25 10:38:37 -0800 | [diff] [blame^] | 1133 | return |
Daniel Norman | 571e182 | 2021-06-25 17:18:25 -0700 | [diff] [blame] | 1134 | |
Daniel Norman | 0374741 | 2022-02-25 10:38:37 -0800 | [diff] [blame^] | 1135 | # TODO(b/192253131): Remove the need for vendor_otatools by fixing |
| 1136 | # backwards-compatibility issues when compiling images across releases. |
| 1137 | if not OPTIONS.vendor_target_files: |
| 1138 | raise ValueError( |
| 1139 | 'Expected vendor_target_files if vendor_otatools is not None.') |
| 1140 | logger.info( |
| 1141 | '%s recompilation will be performed using the vendor otatools.zip', |
| 1142 | partition_img) |
Daniel Norman | 571e182 | 2021-06-25 17:18:25 -0700 | [diff] [blame] | 1143 | |
Daniel Norman | 0374741 | 2022-02-25 10:38:37 -0800 | [diff] [blame^] | 1144 | # Unzip the vendor build's otatools.zip and target-files archive. |
| 1145 | vendor_otatools_dir = common.MakeTempDir( |
| 1146 | prefix='merge_target_files_vendor_otatools_') |
| 1147 | vendor_target_files_dir = common.MakeTempDir( |
| 1148 | prefix='merge_target_files_vendor_target_files_') |
| 1149 | common.UnzipToDir(OPTIONS.vendor_otatools, vendor_otatools_dir) |
| 1150 | common.UnzipToDir(OPTIONS.vendor_target_files, vendor_target_files_dir) |
Daniel Norman | 571e182 | 2021-06-25 17:18:25 -0700 | [diff] [blame] | 1151 | |
Daniel Norman | 0374741 | 2022-02-25 10:38:37 -0800 | [diff] [blame^] | 1152 | # Copy the partition contents from the merged target-files archive to the |
| 1153 | # vendor target-files archive. |
| 1154 | shutil.rmtree(os.path.join(vendor_target_files_dir, partition.upper())) |
| 1155 | shutil.copytree( |
| 1156 | os.path.join(target_files_dir, partition.upper()), |
| 1157 | os.path.join(vendor_target_files_dir, partition.upper()), |
| 1158 | symlinks=True) |
Daniel Norman | 571e182 | 2021-06-25 17:18:25 -0700 | [diff] [blame] | 1159 | |
Daniel Norman | 0374741 | 2022-02-25 10:38:37 -0800 | [diff] [blame^] | 1160 | # Delete then rebuild the partition. |
| 1161 | os.remove(os.path.join(vendor_target_files_dir, 'IMAGES', partition_img)) |
| 1162 | rebuild_partition_command = [ |
| 1163 | os.path.join(vendor_otatools_dir, 'bin', 'add_img_to_target_files'), |
| 1164 | '--verbose', |
| 1165 | '--add_missing', |
| 1166 | ] |
| 1167 | if OPTIONS.rebuild_recovery: |
| 1168 | rebuild_partition_command.append('--rebuild_recovery') |
| 1169 | rebuild_partition_command.append(vendor_target_files_dir) |
| 1170 | logger.info('Recompiling %s: %s', partition_img, |
| 1171 | ' '.join(rebuild_partition_command)) |
| 1172 | common.RunAndCheckOutput(rebuild_partition_command, verbose=True) |
Po Hu | a6c5912 | 2022-02-16 08:41:29 +0000 | [diff] [blame] | 1173 | |
Daniel Norman | 0374741 | 2022-02-25 10:38:37 -0800 | [diff] [blame^] | 1174 | # Move the newly-created image to the merged target files dir. |
| 1175 | if not os.path.exists(os.path.join(target_files_dir, 'IMAGES')): |
| 1176 | os.makedirs(os.path.join(target_files_dir, 'IMAGES')) |
| 1177 | shutil.move( |
| 1178 | os.path.join(vendor_target_files_dir, 'IMAGES', partition_img), |
| 1179 | os.path.join(target_files_dir, 'IMAGES', partition_img)) |
| 1180 | shutil.move( |
| 1181 | os.path.join(vendor_target_files_dir, 'IMAGES', partition_map), |
| 1182 | os.path.join(target_files_dir, 'IMAGES', partition_map)) |
Po Hu | a6c5912 | 2022-02-16 08:41:29 +0000 | [diff] [blame] | 1183 | |
Daniel Norman | 0374741 | 2022-02-25 10:38:37 -0800 | [diff] [blame^] | 1184 | def copy_recovery_file(filename): |
| 1185 | for subdir in ('VENDOR', 'SYSTEM/vendor'): |
| 1186 | source = os.path.join(vendor_target_files_dir, subdir, filename) |
| 1187 | if os.path.exists(source): |
| 1188 | dest = os.path.join(target_files_dir, subdir, filename) |
| 1189 | shutil.copy(source, dest) |
| 1190 | return |
| 1191 | logger.info('Skipping copy_recovery_file for %s, file not found', filename) |
| 1192 | |
| 1193 | if OPTIONS.rebuild_recovery: |
| 1194 | copy_recovery_file('etc/recovery.img') |
| 1195 | copy_recovery_file('bin/install-recovery.sh') |
| 1196 | copy_recovery_file('recovery-from-boot.p') |
Daniel Norman | 571e182 | 2021-06-25 17:18:25 -0700 | [diff] [blame] | 1197 | |
| 1198 | |
Kiyoung Kim | 7cbeda7 | 2019-06-28 13:26:04 +0900 | [diff] [blame] | 1199 | def generate_super_empty_image(target_dir, output_super_empty): |
Tao Bao | 2ad4b82 | 2019-06-27 16:52:12 -0700 | [diff] [blame] | 1200 | """Generates super_empty image from target package. |
Kiyoung Kim | 7cbeda7 | 2019-06-28 13:26:04 +0900 | [diff] [blame] | 1201 | |
| 1202 | Args: |
| 1203 | target_dir: Path to the target file package which contains misc_info.txt for |
| 1204 | detailed information for super image. |
| 1205 | output_super_empty: If provided, copies a super_empty.img file from the |
| 1206 | target files package to this path. |
| 1207 | """ |
Daniel Norman | 1bd2a1d | 2019-04-18 12:32:18 -0700 | [diff] [blame] | 1208 | # Create super_empty.img using the merged misc_info.txt. |
| 1209 | |
Daniel Norman | 4cc9df6 | 2019-07-18 10:11:07 -0700 | [diff] [blame] | 1210 | misc_info_txt = os.path.join(target_dir, 'META', 'misc_info.txt') |
Daniel Norman | 1bd2a1d | 2019-04-18 12:32:18 -0700 | [diff] [blame] | 1211 | |
Kiyoung Kim | ebe7c9c | 2019-06-25 17:09:55 +0900 | [diff] [blame] | 1212 | use_dynamic_partitions = common.LoadDictionaryFromFile(misc_info_txt).get( |
| 1213 | 'use_dynamic_partitions') |
Daniel Norman | 1bd2a1d | 2019-04-18 12:32:18 -0700 | [diff] [blame] | 1214 | |
| 1215 | if use_dynamic_partitions != 'true' and output_super_empty: |
| 1216 | raise ValueError( |
| 1217 | 'Building super_empty.img requires use_dynamic_partitions=true.') |
| 1218 | elif use_dynamic_partitions == 'true': |
Daniel Norman | 4cc9df6 | 2019-07-18 10:11:07 -0700 | [diff] [blame] | 1219 | super_empty_img = os.path.join(target_dir, 'IMAGES', 'super_empty.img') |
Daniel Norman | 1bd2a1d | 2019-04-18 12:32:18 -0700 | [diff] [blame] | 1220 | build_super_image_args = [ |
| 1221 | misc_info_txt, |
| 1222 | super_empty_img, |
| 1223 | ] |
| 1224 | build_super_image.main(build_super_image_args) |
| 1225 | |
| 1226 | # Copy super_empty.img to the user-provided output_super_empty location. |
| 1227 | if output_super_empty: |
| 1228 | shutil.copyfile(super_empty_img, output_super_empty) |
| 1229 | |
Daniel Norman | b8a2f9d | 2019-04-24 12:55:51 -0700 | [diff] [blame] | 1230 | |
Daniel Norman | 0374741 | 2022-02-25 10:38:37 -0800 | [diff] [blame^] | 1231 | def create_target_files_archive(output_zip, source_dir, temp_dir): |
| 1232 | """Creates a target_files zip archive from the input source dir. |
Kiyoung Kim | 7cbeda7 | 2019-06-28 13:26:04 +0900 | [diff] [blame] | 1233 | |
| 1234 | Args: |
Daniel Norman | 0374741 | 2022-02-25 10:38:37 -0800 | [diff] [blame^] | 1235 | output_zip: The name of the zip archive target files package. |
Kiyoung Kim | 7cbeda7 | 2019-06-28 13:26:04 +0900 | [diff] [blame] | 1236 | source_dir: The target directory contains package to be archived. |
| 1237 | temp_dir: Path to temporary directory for any intermediate files. |
| 1238 | """ |
Bill Peckham | e9eb5f9 | 2019-02-01 15:52:10 -0800 | [diff] [blame] | 1239 | output_target_files_list = os.path.join(temp_dir, 'output.list') |
Daniel Norman | 4cc9df6 | 2019-07-18 10:11:07 -0700 | [diff] [blame] | 1240 | output_target_files_meta_dir = os.path.join(source_dir, 'META') |
Bill Peckham | e9eb5f9 | 2019-02-01 15:52:10 -0800 | [diff] [blame] | 1241 | |
Daniel Norman | dbbf5a3 | 2020-10-22 16:03:32 -0700 | [diff] [blame] | 1242 | def files_from_path(target_path, extra_args=None): |
| 1243 | """Gets files under the given path and return a sorted list.""" |
| 1244 | find_command = ['find', target_path] + (extra_args or []) |
| 1245 | find_process = common.Run( |
| 1246 | find_command, stdout=subprocess.PIPE, verbose=False) |
| 1247 | return common.RunAndCheckOutput(['sort'], |
| 1248 | stdin=find_process.stdout, |
| 1249 | verbose=False) |
| 1250 | |
Daniel Norman | 0374741 | 2022-02-25 10:38:37 -0800 | [diff] [blame^] | 1251 | # META content appears first in the zip. This is done by the |
| 1252 | # standard build system for optimized extraction of those files, |
| 1253 | # so we do the same step for merged target_files.zips here too. |
Kiyoung Kim | 7cbeda7 | 2019-06-28 13:26:04 +0900 | [diff] [blame] | 1254 | meta_content = files_from_path(output_target_files_meta_dir) |
Daniel Norman | 4cc9df6 | 2019-07-18 10:11:07 -0700 | [diff] [blame] | 1255 | other_content = files_from_path( |
| 1256 | source_dir, |
| 1257 | ['-path', output_target_files_meta_dir, '-prune', '-o', '-print']) |
Bill Peckham | e9eb5f9 | 2019-02-01 15:52:10 -0800 | [diff] [blame] | 1258 | |
Tao Bao | 2ad4b82 | 2019-06-27 16:52:12 -0700 | [diff] [blame] | 1259 | with open(output_target_files_list, 'w') as f: |
Bill Peckham | e9eb5f9 | 2019-02-01 15:52:10 -0800 | [diff] [blame] | 1260 | f.write(meta_content) |
| 1261 | f.write(other_content) |
| 1262 | |
| 1263 | command = [ |
Bill Peckham | f753e15 | 2019-02-19 18:02:46 -0800 | [diff] [blame] | 1264 | 'soong_zip', |
Bill Peckham | e9eb5f9 | 2019-02-01 15:52:10 -0800 | [diff] [blame] | 1265 | '-d', |
Daniel Norman | e5b134a | 2019-04-17 14:54:06 -0700 | [diff] [blame] | 1266 | '-o', |
Daniel Norman | 0374741 | 2022-02-25 10:38:37 -0800 | [diff] [blame^] | 1267 | os.path.abspath(output_zip), |
Daniel Norman | e5b134a | 2019-04-17 14:54:06 -0700 | [diff] [blame] | 1268 | '-C', |
Kiyoung Kim | 7cbeda7 | 2019-06-28 13:26:04 +0900 | [diff] [blame] | 1269 | source_dir, |
Daniel Norman | eaf5c1d | 2021-02-09 11:01:42 -0800 | [diff] [blame] | 1270 | '-r', |
Daniel Norman | e5b134a | 2019-04-17 14:54:06 -0700 | [diff] [blame] | 1271 | output_target_files_list, |
Bill Peckham | e9eb5f9 | 2019-02-01 15:52:10 -0800 | [diff] [blame] | 1272 | ] |
Kiyoung Kim | 7cbeda7 | 2019-06-28 13:26:04 +0900 | [diff] [blame] | 1273 | |
Daniel Norman | 0374741 | 2022-02-25 10:38:37 -0800 | [diff] [blame^] | 1274 | logger.info('creating %s', output_zip) |
Daniel Norman | eaf5c1d | 2021-02-09 11:01:42 -0800 | [diff] [blame] | 1275 | common.RunAndCheckOutput(command, verbose=True) |
Daniel Norman | 0374741 | 2022-02-25 10:38:37 -0800 | [diff] [blame^] | 1276 | logger.info('finished creating %s', output_zip) |
Kiyoung Kim | 7cbeda7 | 2019-06-28 13:26:04 +0900 | [diff] [blame] | 1277 | |
| 1278 | |
Daniel Norman | 0374741 | 2022-02-25 10:38:37 -0800 | [diff] [blame^] | 1279 | def merge_target_files(temp_dir): |
Tao Bao | 2ad4b82 | 2019-06-27 16:52:12 -0700 | [diff] [blame] | 1280 | """Merges two target files packages together. |
Kiyoung Kim | 7cbeda7 | 2019-06-28 13:26:04 +0900 | [diff] [blame] | 1281 | |
Daniel Norman | 0374741 | 2022-02-25 10:38:37 -0800 | [diff] [blame^] | 1282 | This function uses framework and vendor target files packages as input, |
Kiyoung Kim | 7cbeda7 | 2019-06-28 13:26:04 +0900 | [diff] [blame] | 1283 | performs various file extractions, special case processing, and finally |
| 1284 | creates a merged zip archive as output. |
| 1285 | |
| 1286 | Args: |
| 1287 | temp_dir: The name of a directory we use when we extract items from the |
| 1288 | input target files packages, and also a scratch directory that we use for |
| 1289 | temporary files. |
Kiyoung Kim | 7cbeda7 | 2019-06-28 13:26:04 +0900 | [diff] [blame] | 1290 | """ |
| 1291 | |
| 1292 | logger.info('starting: merge framework %s and vendor %s into output %s', |
Daniel Norman | 0374741 | 2022-02-25 10:38:37 -0800 | [diff] [blame^] | 1293 | OPTIONS.framework_target_files, OPTIONS.vendor_target_files, |
| 1294 | OPTIONS.output_target_files) |
Kiyoung Kim | 7cbeda7 | 2019-06-28 13:26:04 +0900 | [diff] [blame] | 1295 | |
Daniel Norman | 0374741 | 2022-02-25 10:38:37 -0800 | [diff] [blame^] | 1296 | output_target_files_temp_dir = create_merged_package(temp_dir) |
Kiyoung Kim | 7cbeda7 | 2019-06-28 13:26:04 +0900 | [diff] [blame] | 1297 | |
Yifan Hong | ade0d3f | 2019-08-21 16:37:11 -0700 | [diff] [blame] | 1298 | if not check_target_files_vintf.CheckVintf(output_target_files_temp_dir): |
Daniel Norman | b0c7591 | 2020-09-24 14:30:21 -0700 | [diff] [blame] | 1299 | raise RuntimeError('Incompatible VINTF metadata') |
Yifan Hong | ade0d3f | 2019-08-21 16:37:11 -0700 | [diff] [blame] | 1300 | |
Daniel Norman | 21c34f7 | 2020-11-11 17:25:50 -0800 | [diff] [blame] | 1301 | partition_map = common.PartitionMapFromTargetFiles( |
| 1302 | output_target_files_temp_dir) |
| 1303 | |
Daniel Norman | d335156 | 2020-10-29 12:33:11 -0700 | [diff] [blame] | 1304 | # Generate and check for cross-partition violations of sharedUserId |
| 1305 | # values in APKs. This requires the input target-files packages to contain |
| 1306 | # *.apk files. |
Daniel Norman | b8d52a2 | 2020-10-26 17:55:00 -0700 | [diff] [blame] | 1307 | shareduid_violation_modules = os.path.join( |
| 1308 | output_target_files_temp_dir, 'META', 'shareduid_violation_modules.json') |
| 1309 | with open(shareduid_violation_modules, 'w') as f: |
Daniel Norman | b8d52a2 | 2020-10-26 17:55:00 -0700 | [diff] [blame] | 1310 | violation = find_shareduid_violation.FindShareduidViolation( |
| 1311 | output_target_files_temp_dir, partition_map) |
Daniel Norman | d335156 | 2020-10-29 12:33:11 -0700 | [diff] [blame] | 1312 | |
| 1313 | # Write the output to a file to enable debugging. |
Daniel Norman | b8d52a2 | 2020-10-26 17:55:00 -0700 | [diff] [blame] | 1314 | f.write(violation) |
Daniel Norman | d335156 | 2020-10-29 12:33:11 -0700 | [diff] [blame] | 1315 | |
| 1316 | # Check for violations across the input builds' partition groups. |
| 1317 | shareduid_errors = common.SharedUidPartitionViolations( |
Daniel Norman | 0374741 | 2022-02-25 10:38:37 -0800 | [diff] [blame^] | 1318 | json.loads(violation), |
| 1319 | [OPTIONS.framework_partition_set, OPTIONS.vendor_partition_set]) |
Daniel Norman | d335156 | 2020-10-29 12:33:11 -0700 | [diff] [blame] | 1320 | if shareduid_errors: |
| 1321 | for error in shareduid_errors: |
| 1322 | logger.error(error) |
| 1323 | raise ValueError('sharedUserId APK error. See %s' % |
| 1324 | shareduid_violation_modules) |
Daniel Norman | b8d52a2 | 2020-10-26 17:55:00 -0700 | [diff] [blame] | 1325 | |
Daniel Norman | 48603ff | 2021-02-22 15:15:24 -0800 | [diff] [blame] | 1326 | # host_init_verifier and secilc check only the following partitions: |
Daniel Norman | 21c34f7 | 2020-11-11 17:25:50 -0800 | [diff] [blame] | 1327 | filtered_partitions = { |
| 1328 | partition: path |
| 1329 | for partition, path in partition_map.items() |
Daniel Norman | 21c34f7 | 2020-11-11 17:25:50 -0800 | [diff] [blame] | 1330 | if partition in ['system', 'system_ext', 'product', 'vendor', 'odm'] |
| 1331 | } |
Daniel Norman | 48603ff | 2021-02-22 15:15:24 -0800 | [diff] [blame] | 1332 | |
| 1333 | # Run host_init_verifier on the combined init rc files. |
Daniel Norman | 21c34f7 | 2020-11-11 17:25:50 -0800 | [diff] [blame] | 1334 | common.RunHostInitVerifier( |
| 1335 | product_out=output_target_files_temp_dir, |
| 1336 | partition_map=filtered_partitions) |
| 1337 | |
Daniel Norman | 48603ff | 2021-02-22 15:15:24 -0800 | [diff] [blame] | 1338 | # Check that the split sepolicy from the multiple builds can compile. |
Daniel Norman | 571e182 | 2021-06-25 17:18:25 -0700 | [diff] [blame] | 1339 | split_sepolicy_cmd = compile_split_sepolicy(output_target_files_temp_dir, |
| 1340 | filtered_partitions) |
Daniel Norman | 48603ff | 2021-02-22 15:15:24 -0800 | [diff] [blame] | 1341 | logger.info('Compiling split sepolicy: %s', ' '.join(split_sepolicy_cmd)) |
| 1342 | common.RunAndCheckOutput(split_sepolicy_cmd) |
Daniel Norman | 571e182 | 2021-06-25 17:18:25 -0700 | [diff] [blame] | 1343 | # Include the compiled policy in an image if requested. |
Daniel Norman | 0374741 | 2022-02-25 10:38:37 -0800 | [diff] [blame^] | 1344 | if OPTIONS.rebuild_sepolicy: |
| 1345 | rebuild_image_with_sepolicy(output_target_files_temp_dir) |
Daniel Norman | 48603ff | 2021-02-22 15:15:24 -0800 | [diff] [blame] | 1346 | |
Daniel Norman | e9af70a | 2021-04-15 16:39:22 -0700 | [diff] [blame] | 1347 | # Run validation checks on the pre-installed APEX files. |
| 1348 | validate_merged_apex_info(output_target_files_temp_dir, partition_map.keys()) |
| 1349 | |
Daniel Norman | 0374741 | 2022-02-25 10:38:37 -0800 | [diff] [blame^] | 1350 | generate_missing_images(output_target_files_temp_dir) |
Kiyoung Kim | 7cbeda7 | 2019-06-28 13:26:04 +0900 | [diff] [blame] | 1351 | |
Daniel Norman | 0374741 | 2022-02-25 10:38:37 -0800 | [diff] [blame^] | 1352 | generate_super_empty_image(output_target_files_temp_dir, |
| 1353 | OPTIONS.output_super_empty) |
Kiyoung Kim | 7cbeda7 | 2019-06-28 13:26:04 +0900 | [diff] [blame] | 1354 | |
Kiyoung Kim | 7cbeda7 | 2019-06-28 13:26:04 +0900 | [diff] [blame] | 1355 | # Finally, create the output target files zip archive and/or copy the |
| 1356 | # output items to the output target files directory. |
| 1357 | |
Daniel Norman | 0374741 | 2022-02-25 10:38:37 -0800 | [diff] [blame^] | 1358 | if OPTIONS.output_dir: |
| 1359 | copy_items(output_target_files_temp_dir, OPTIONS.output_dir, |
| 1360 | OPTIONS.output_item_list) |
Kiyoung Kim | 7cbeda7 | 2019-06-28 13:26:04 +0900 | [diff] [blame] | 1361 | |
Daniel Norman | 0374741 | 2022-02-25 10:38:37 -0800 | [diff] [blame^] | 1362 | if not OPTIONS.output_target_files: |
Kiyoung Kim | 7cbeda7 | 2019-06-28 13:26:04 +0900 | [diff] [blame] | 1363 | return |
| 1364 | |
Daniel Norman | 0374741 | 2022-02-25 10:38:37 -0800 | [diff] [blame^] | 1365 | # Create the merged META/care_map.pb if the device uses A/B updates. |
| 1366 | if OPTIONS.merged_misc_info['ab_update'] == 'true': |
Iavor-Valentin Iftime | b837b71 | 2022-01-27 16:29:37 +0000 | [diff] [blame] | 1367 | generate_care_map(partition_map.keys(), output_target_files_temp_dir) |
Daniel Norman | db8cacc | 2021-04-09 15:34:43 -0700 | [diff] [blame] | 1368 | |
Daniel Norman | 0374741 | 2022-02-25 10:38:37 -0800 | [diff] [blame^] | 1369 | create_target_files_archive(OPTIONS.output_target_files, |
| 1370 | output_target_files_temp_dir, temp_dir) |
Bill Peckham | e9eb5f9 | 2019-02-01 15:52:10 -0800 | [diff] [blame] | 1371 | |
Daniel Norman | 74eb74b | 2019-09-18 14:01:48 -0700 | [diff] [blame] | 1372 | # Create the IMG package from the merged target files package. |
Daniel Norman | 0374741 | 2022-02-25 10:38:37 -0800 | [diff] [blame^] | 1373 | if OPTIONS.output_img: |
| 1374 | img_from_target_files.main( |
| 1375 | [OPTIONS.output_target_files, OPTIONS.output_img]) |
Daniel Norman | 74eb74b | 2019-09-18 14:01:48 -0700 | [diff] [blame] | 1376 | |
Daniel Norman | 3b64ce1 | 2019-04-16 16:11:35 -0700 | [diff] [blame] | 1377 | # Create the OTA package from the merged target files package. |
| 1378 | |
Daniel Norman | 0374741 | 2022-02-25 10:38:37 -0800 | [diff] [blame^] | 1379 | if OPTIONS.output_ota: |
| 1380 | ota_from_target_files.main( |
| 1381 | [OPTIONS.output_target_files, OPTIONS.output_ota]) |
Daniel Norman | 3b64ce1 | 2019-04-16 16:11:35 -0700 | [diff] [blame] | 1382 | |
Daniel Norman | 1bd2a1d | 2019-04-18 12:32:18 -0700 | [diff] [blame] | 1383 | |
Daniel Norman | 2c99c5b | 2019-03-07 13:01:48 -0800 | [diff] [blame] | 1384 | def call_func_with_temp_dir(func, keep_tmp): |
Tao Bao | 2ad4b82 | 2019-06-27 16:52:12 -0700 | [diff] [blame] | 1385 | """Manages the creation and cleanup of the temporary directory. |
Bill Peckham | e9eb5f9 | 2019-02-01 15:52:10 -0800 | [diff] [blame] | 1386 | |
Daniel Norman | 2c99c5b | 2019-03-07 13:01:48 -0800 | [diff] [blame] | 1387 | This function calls the given function after first creating a temporary |
Bill Peckham | e9eb5f9 | 2019-02-01 15:52:10 -0800 | [diff] [blame] | 1388 | directory. It also cleans up the temporary directory. |
| 1389 | |
| 1390 | Args: |
Daniel Norman | e5b134a | 2019-04-17 14:54:06 -0700 | [diff] [blame] | 1391 | func: The function to call. Should accept one parameter, the path to the |
| 1392 | temporary directory. |
Bill Peckham | e9eb5f9 | 2019-02-01 15:52:10 -0800 | [diff] [blame] | 1393 | keep_tmp: Keep the temporary directory after processing is complete. |
Bill Peckham | e9eb5f9 | 2019-02-01 15:52:10 -0800 | [diff] [blame] | 1394 | """ |
| 1395 | |
| 1396 | # Create a temporary directory. This will serve as the parent of directories |
| 1397 | # we use when we extract items from the input target files packages, and also |
| 1398 | # a scratch directory that we use for temporary files. |
| 1399 | |
Bill Peckham | e9eb5f9 | 2019-02-01 15:52:10 -0800 | [diff] [blame] | 1400 | temp_dir = common.MakeTempDir(prefix='merge_target_files_') |
| 1401 | |
| 1402 | try: |
Daniel Norman | 2c99c5b | 2019-03-07 13:01:48 -0800 | [diff] [blame] | 1403 | func(temp_dir) |
Bill Peckham | e9eb5f9 | 2019-02-01 15:52:10 -0800 | [diff] [blame] | 1404 | finally: |
| 1405 | if keep_tmp: |
| 1406 | logger.info('keeping %s', temp_dir) |
| 1407 | else: |
| 1408 | common.Cleanup() |
| 1409 | |
| 1410 | |
| 1411 | def main(): |
| 1412 | """The main function. |
| 1413 | |
Daniel Norman | 2c99c5b | 2019-03-07 13:01:48 -0800 | [diff] [blame] | 1414 | Process command line arguments, then call merge_target_files to |
Bill Peckham | e9eb5f9 | 2019-02-01 15:52:10 -0800 | [diff] [blame] | 1415 | perform the heavy lifting. |
Bill Peckham | e9eb5f9 | 2019-02-01 15:52:10 -0800 | [diff] [blame] | 1416 | """ |
| 1417 | |
| 1418 | common.InitLogging() |
| 1419 | |
Bill Peckham | f753e15 | 2019-02-19 18:02:46 -0800 | [diff] [blame] | 1420 | def option_handler(o, a): |
| 1421 | if o == '--system-target-files': |
Daniel Norman | d5d70ea | 2019-06-05 15:13:43 -0700 | [diff] [blame] | 1422 | logger.warning( |
| 1423 | '--system-target-files has been renamed to --framework-target-files') |
| 1424 | OPTIONS.framework_target_files = a |
| 1425 | elif o == '--framework-target-files': |
| 1426 | OPTIONS.framework_target_files = a |
Daniel Norman | 2c99c5b | 2019-03-07 13:01:48 -0800 | [diff] [blame] | 1427 | elif o == '--system-item-list': |
Daniel Norman | d5d70ea | 2019-06-05 15:13:43 -0700 | [diff] [blame] | 1428 | logger.warning( |
| 1429 | '--system-item-list has been renamed to --framework-item-list') |
| 1430 | OPTIONS.framework_item_list = a |
| 1431 | elif o == '--framework-item-list': |
| 1432 | OPTIONS.framework_item_list = a |
Daniel Norman | 2c99c5b | 2019-03-07 13:01:48 -0800 | [diff] [blame] | 1433 | elif o == '--system-misc-info-keys': |
Daniel Norman | 4cc9df6 | 2019-07-18 10:11:07 -0700 | [diff] [blame] | 1434 | logger.warning('--system-misc-info-keys has been renamed to ' |
| 1435 | '--framework-misc-info-keys') |
Daniel Norman | d5d70ea | 2019-06-05 15:13:43 -0700 | [diff] [blame] | 1436 | OPTIONS.framework_misc_info_keys = a |
| 1437 | elif o == '--framework-misc-info-keys': |
| 1438 | OPTIONS.framework_misc_info_keys = a |
Bill Peckham | f753e15 | 2019-02-19 18:02:46 -0800 | [diff] [blame] | 1439 | elif o == '--other-target-files': |
Daniel Norman | d5d70ea | 2019-06-05 15:13:43 -0700 | [diff] [blame] | 1440 | logger.warning( |
| 1441 | '--other-target-files has been renamed to --vendor-target-files') |
| 1442 | OPTIONS.vendor_target_files = a |
| 1443 | elif o == '--vendor-target-files': |
| 1444 | OPTIONS.vendor_target_files = a |
Daniel Norman | 2c99c5b | 2019-03-07 13:01:48 -0800 | [diff] [blame] | 1445 | elif o == '--other-item-list': |
Daniel Norman | 2d7989a | 2021-04-05 17:40:47 +0000 | [diff] [blame] | 1446 | logger.warning('--other-item-list has been renamed to --vendor-item-list') |
Daniel Norman | d5d70ea | 2019-06-05 15:13:43 -0700 | [diff] [blame] | 1447 | OPTIONS.vendor_item_list = a |
| 1448 | elif o == '--vendor-item-list': |
| 1449 | OPTIONS.vendor_item_list = a |
Bill Peckham | f753e15 | 2019-02-19 18:02:46 -0800 | [diff] [blame] | 1450 | elif o == '--output-target-files': |
| 1451 | OPTIONS.output_target_files = a |
Daniel Norman | fdb3881 | 2019-04-15 09:47:24 -0700 | [diff] [blame] | 1452 | elif o == '--output-dir': |
| 1453 | OPTIONS.output_dir = a |
| 1454 | elif o == '--output-item-list': |
| 1455 | OPTIONS.output_item_list = a |
Daniel Norman | 3b64ce1 | 2019-04-16 16:11:35 -0700 | [diff] [blame] | 1456 | elif o == '--output-ota': |
| 1457 | OPTIONS.output_ota = a |
Daniel Norman | 1bd2a1d | 2019-04-18 12:32:18 -0700 | [diff] [blame] | 1458 | elif o == '--output-img': |
| 1459 | OPTIONS.output_img = a |
Daniel Norman | f031825 | 2019-04-15 11:34:56 -0700 | [diff] [blame] | 1460 | elif o == '--output-super-empty': |
| 1461 | OPTIONS.output_super_empty = a |
Po Hu | a6c5912 | 2022-02-16 08:41:29 +0000 | [diff] [blame] | 1462 | elif o == '--rebuild_recovery': |
Daniel Norman | a4911da | 2019-03-15 14:36:21 -0700 | [diff] [blame] | 1463 | OPTIONS.rebuild_recovery = True |
Daniel Norman | b0c7591 | 2020-09-24 14:30:21 -0700 | [diff] [blame] | 1464 | elif o == '--allow-duplicate-apkapex-keys': |
| 1465 | OPTIONS.allow_duplicate_apkapex_keys = True |
Daniel Norman | 571e182 | 2021-06-25 17:18:25 -0700 | [diff] [blame] | 1466 | elif o == '--vendor-otatools': |
| 1467 | OPTIONS.vendor_otatools = a |
| 1468 | elif o == '--rebuild-sepolicy': |
| 1469 | OPTIONS.rebuild_sepolicy = True |
Bill Peckham | 364c1cc | 2019-03-29 18:27:23 -0700 | [diff] [blame] | 1470 | elif o == '--keep-tmp': |
Bill Peckham | f753e15 | 2019-02-19 18:02:46 -0800 | [diff] [blame] | 1471 | OPTIONS.keep_tmp = True |
Jose Galmes | 9c8f6eb | 2021-07-21 09:34:08 -0700 | [diff] [blame] | 1472 | elif o == '--framework-dexpreopt-config': |
| 1473 | OPTIONS.framework_dexpreopt_config = a |
| 1474 | elif o == '--framework-dexpreopt-tools': |
| 1475 | OPTIONS.framework_dexpreopt_tools = a |
| 1476 | elif o == '--vendor-dexpreopt-config': |
| 1477 | OPTIONS.vendor_dexpreopt_config = a |
Bill Peckham | f753e15 | 2019-02-19 18:02:46 -0800 | [diff] [blame] | 1478 | else: |
| 1479 | return False |
| 1480 | return True |
Bill Peckham | e9eb5f9 | 2019-02-01 15:52:10 -0800 | [diff] [blame] | 1481 | |
Bill Peckham | f753e15 | 2019-02-19 18:02:46 -0800 | [diff] [blame] | 1482 | args = common.ParseOptions( |
Daniel Norman | e5b134a | 2019-04-17 14:54:06 -0700 | [diff] [blame] | 1483 | sys.argv[1:], |
| 1484 | __doc__, |
Bill Peckham | f753e15 | 2019-02-19 18:02:46 -0800 | [diff] [blame] | 1485 | extra_long_opts=[ |
| 1486 | 'system-target-files=', |
Daniel Norman | d5d70ea | 2019-06-05 15:13:43 -0700 | [diff] [blame] | 1487 | 'framework-target-files=', |
Daniel Norman | 2c99c5b | 2019-03-07 13:01:48 -0800 | [diff] [blame] | 1488 | 'system-item-list=', |
Daniel Norman | d5d70ea | 2019-06-05 15:13:43 -0700 | [diff] [blame] | 1489 | 'framework-item-list=', |
Daniel Norman | 2c99c5b | 2019-03-07 13:01:48 -0800 | [diff] [blame] | 1490 | 'system-misc-info-keys=', |
Daniel Norman | d5d70ea | 2019-06-05 15:13:43 -0700 | [diff] [blame] | 1491 | 'framework-misc-info-keys=', |
Bill Peckham | f753e15 | 2019-02-19 18:02:46 -0800 | [diff] [blame] | 1492 | 'other-target-files=', |
Daniel Norman | d5d70ea | 2019-06-05 15:13:43 -0700 | [diff] [blame] | 1493 | 'vendor-target-files=', |
Daniel Norman | 2c99c5b | 2019-03-07 13:01:48 -0800 | [diff] [blame] | 1494 | 'other-item-list=', |
Daniel Norman | d5d70ea | 2019-06-05 15:13:43 -0700 | [diff] [blame] | 1495 | 'vendor-item-list=', |
Bill Peckham | f753e15 | 2019-02-19 18:02:46 -0800 | [diff] [blame] | 1496 | 'output-target-files=', |
Daniel Norman | fdb3881 | 2019-04-15 09:47:24 -0700 | [diff] [blame] | 1497 | 'output-dir=', |
| 1498 | 'output-item-list=', |
Daniel Norman | 3b64ce1 | 2019-04-16 16:11:35 -0700 | [diff] [blame] | 1499 | 'output-ota=', |
Daniel Norman | 1bd2a1d | 2019-04-18 12:32:18 -0700 | [diff] [blame] | 1500 | 'output-img=', |
Daniel Norman | f031825 | 2019-04-15 11:34:56 -0700 | [diff] [blame] | 1501 | 'output-super-empty=', |
Jose Galmes | 9c8f6eb | 2021-07-21 09:34:08 -0700 | [diff] [blame] | 1502 | 'framework-dexpreopt-config=', |
| 1503 | 'framework-dexpreopt-tools=', |
| 1504 | 'vendor-dexpreopt-config=', |
Daniel Norman | a4911da | 2019-03-15 14:36:21 -0700 | [diff] [blame] | 1505 | 'rebuild_recovery', |
Daniel Norman | b0c7591 | 2020-09-24 14:30:21 -0700 | [diff] [blame] | 1506 | 'allow-duplicate-apkapex-keys', |
Daniel Norman | 571e182 | 2021-06-25 17:18:25 -0700 | [diff] [blame] | 1507 | 'vendor-otatools=', |
| 1508 | 'rebuild-sepolicy', |
Bill Peckham | 364c1cc | 2019-03-29 18:27:23 -0700 | [diff] [blame] | 1509 | 'keep-tmp', |
Bill Peckham | f753e15 | 2019-02-19 18:02:46 -0800 | [diff] [blame] | 1510 | ], |
| 1511 | extra_option_handler=option_handler) |
Bill Peckham | e9eb5f9 | 2019-02-01 15:52:10 -0800 | [diff] [blame] | 1512 | |
Tao Bao | 2ad4b82 | 2019-06-27 16:52:12 -0700 | [diff] [blame] | 1513 | # pylint: disable=too-many-boolean-expressions |
Daniel Norman | d5d70ea | 2019-06-05 15:13:43 -0700 | [diff] [blame] | 1514 | if (args or OPTIONS.framework_target_files is None or |
| 1515 | OPTIONS.vendor_target_files is None or |
Daniel Norman | e5b134a | 2019-04-17 14:54:06 -0700 | [diff] [blame] | 1516 | (OPTIONS.output_target_files is None and OPTIONS.output_dir is None) or |
Po Hu | a6c5912 | 2022-02-16 08:41:29 +0000 | [diff] [blame] | 1517 | (OPTIONS.output_dir is not None and OPTIONS.output_item_list is None) or |
| 1518 | (OPTIONS.rebuild_recovery and not OPTIONS.rebuild_sepolicy)): |
Bill Peckham | f753e15 | 2019-02-19 18:02:46 -0800 | [diff] [blame] | 1519 | common.Usage(__doc__) |
Bill Peckham | 889b0c6 | 2019-02-21 18:53:37 -0800 | [diff] [blame] | 1520 | sys.exit(1) |
Bill Peckham | e9eb5f9 | 2019-02-01 15:52:10 -0800 | [diff] [blame] | 1521 | |
Daniel Norman | d5d70ea | 2019-06-05 15:13:43 -0700 | [diff] [blame] | 1522 | if OPTIONS.framework_item_list: |
Daniel Norman | 0374741 | 2022-02-25 10:38:37 -0800 | [diff] [blame^] | 1523 | OPTIONS.framework_item_list = common.LoadListFromFile( |
| 1524 | OPTIONS.framework_item_list) |
Daniel Norman | 2c99c5b | 2019-03-07 13:01:48 -0800 | [diff] [blame] | 1525 | else: |
Daniel Norman | 0374741 | 2022-02-25 10:38:37 -0800 | [diff] [blame^] | 1526 | OPTIONS.framework_item_list = DEFAULT_FRAMEWORK_ITEM_LIST |
| 1527 | OPTIONS.framework_partition_set = item_list_to_partition_set( |
| 1528 | OPTIONS.framework_item_list) |
Daniel Norman | 2c99c5b | 2019-03-07 13:01:48 -0800 | [diff] [blame] | 1529 | |
Daniel Norman | d5d70ea | 2019-06-05 15:13:43 -0700 | [diff] [blame] | 1530 | if OPTIONS.framework_misc_info_keys: |
Daniel Norman | 0374741 | 2022-02-25 10:38:37 -0800 | [diff] [blame^] | 1531 | OPTIONS.framework_misc_info_keys = common.LoadListFromFile( |
Daniel Norman | d5d70ea | 2019-06-05 15:13:43 -0700 | [diff] [blame] | 1532 | OPTIONS.framework_misc_info_keys) |
Daniel Norman | 2c99c5b | 2019-03-07 13:01:48 -0800 | [diff] [blame] | 1533 | else: |
Daniel Norman | 0374741 | 2022-02-25 10:38:37 -0800 | [diff] [blame^] | 1534 | OPTIONS.framework_misc_info_keys = DEFAULT_FRAMEWORK_MISC_INFO_KEYS |
Daniel Norman | 2c99c5b | 2019-03-07 13:01:48 -0800 | [diff] [blame] | 1535 | |
Daniel Norman | d5d70ea | 2019-06-05 15:13:43 -0700 | [diff] [blame] | 1536 | if OPTIONS.vendor_item_list: |
Daniel Norman | 0374741 | 2022-02-25 10:38:37 -0800 | [diff] [blame^] | 1537 | OPTIONS.vendor_item_list = common.LoadListFromFile(OPTIONS.vendor_item_list) |
Daniel Norman | 2c99c5b | 2019-03-07 13:01:48 -0800 | [diff] [blame] | 1538 | else: |
Daniel Norman | 0374741 | 2022-02-25 10:38:37 -0800 | [diff] [blame^] | 1539 | OPTIONS.vendor_item_list = DEFAULT_VENDOR_ITEM_LIST |
| 1540 | OPTIONS.vendor_partition_set = item_list_to_partition_set( |
| 1541 | OPTIONS.vendor_item_list) |
Daniel Norman | 2c99c5b | 2019-03-07 13:01:48 -0800 | [diff] [blame] | 1542 | |
Daniel Norman | fdb3881 | 2019-04-15 09:47:24 -0700 | [diff] [blame] | 1543 | if OPTIONS.output_item_list: |
Daniel Norman | 0374741 | 2022-02-25 10:38:37 -0800 | [diff] [blame^] | 1544 | OPTIONS.output_item_list = common.LoadListFromFile(OPTIONS.output_item_list) |
Daniel Norman | fdb3881 | 2019-04-15 09:47:24 -0700 | [diff] [blame] | 1545 | else: |
Daniel Norman | 0374741 | 2022-02-25 10:38:37 -0800 | [diff] [blame^] | 1546 | OPTIONS.output_item_list = None |
Daniel Norman | fdb3881 | 2019-04-15 09:47:24 -0700 | [diff] [blame] | 1547 | |
Daniel Norman | 0374741 | 2022-02-25 10:38:37 -0800 | [diff] [blame^] | 1548 | if not validate_config_lists(): |
Daniel Norman | e596452 | 2019-03-19 10:32:03 -0700 | [diff] [blame] | 1549 | sys.exit(1) |
| 1550 | |
Daniel Norman | 0374741 | 2022-02-25 10:38:37 -0800 | [diff] [blame^] | 1551 | call_func_with_temp_dir(lambda temp_dir: merge_target_files(temp_dir), |
| 1552 | OPTIONS.keep_tmp) |
Bill Peckham | e9eb5f9 | 2019-02-01 15:52:10 -0800 | [diff] [blame] | 1553 | |
| 1554 | |
| 1555 | if __name__ == '__main__': |
Bill Peckham | 889b0c6 | 2019-02-21 18:53:37 -0800 | [diff] [blame] | 1556 | main() |