blob: dc75ce2605fb03e8158e5773f65d32eb2251bfa8 [file] [log] [blame]
Doug Zongkereef39442009-04-02 12:14:19 -07001#!/usr/bin/env python
2#
3# Copyright (C) 2008 The Android Open Source Project
4#
5# Licensed under the Apache License, Version 2.0 (the "License");
6# you may not use this file except in compliance with the License.
7# You may obtain a copy of the License at
8#
9# http://www.apache.org/licenses/LICENSE-2.0
10#
11# Unless required by applicable law or agreed to in writing, software
12# distributed under the License is distributed on an "AS IS" BASIS,
13# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
14# See the License for the specific language governing permissions and
15# limitations under the License.
16
17"""
Tao Bao30df8b42018-04-23 15:32:53 -070018Given a target-files zipfile, produces an OTA package that installs that build.
19An incremental OTA is produced if -i is given, otherwise a full OTA is produced.
Doug Zongkereef39442009-04-02 12:14:19 -070020
Tao Bao30df8b42018-04-23 15:32:53 -070021Usage: ota_from_target_files [options] input_target_files output_ota_package
Doug Zongkereef39442009-04-02 12:14:19 -070022
Tao Bao30df8b42018-04-23 15:32:53 -070023Common options that apply to both of non-A/B and A/B OTAs
24
25 --downgrade
26 Intentionally generate an incremental OTA that updates from a newer build
Tao Baofaa8e0b2018-04-12 14:31:43 -070027 to an older one (e.g. downgrading from P preview back to O MR1).
28 "ota-downgrade=yes" will be set in the package metadata file. A data wipe
29 will always be enforced when using this flag, so "ota-wipe=yes" will also
30 be included in the metadata file. The update-binary in the source build
31 will be used in the OTA package, unless --binary flag is specified. Please
32 also check the comment for --override_timestamp below.
Tao Bao30df8b42018-04-23 15:32:53 -070033
34 -i (--incremental_from) <file>
35 Generate an incremental OTA using the given target-files zip as the
36 starting build.
37
38 -k (--package_key) <key>
39 Key to use to sign the package (default is the value of
40 default_system_dev_certificate from the input target-files's
Tao Bao59cf0c52019-06-25 10:04:24 -070041 META/misc_info.txt, or "build/make/target/product/security/testkey" if
42 that value is not specified).
Doug Zongkerafb32ea2011-09-22 10:28:04 -070043
44 For incremental OTAs, the default value is based on the source
45 target-file, not the target build.
Doug Zongkereef39442009-04-02 12:14:19 -070046
Tao Bao30df8b42018-04-23 15:32:53 -070047 --override_timestamp
48 Intentionally generate an incremental OTA that updates from a newer build
Tao Baofaa8e0b2018-04-12 14:31:43 -070049 to an older one (based on timestamp comparison), by setting the downgrade
50 flag in the package metadata. This differs from --downgrade flag, as we
51 don't enforce a data wipe with this flag. Because we know for sure this is
52 NOT an actual downgrade case, but two builds happen to be cut in a reverse
53 order (e.g. from two branches). A legit use case is that we cut a new
54 build C (after having A and B), but want to enfore an update path of A ->
55 C -> B. Specifying --downgrade may not help since that would enforce a
56 data wipe for C -> B update.
57
58 We used to set a fake timestamp in the package metadata for this flow. But
59 now we consolidate the two cases (i.e. an actual downgrade, or a downgrade
60 based on timestamp) with the same "ota-downgrade=yes" flag, with the
61 difference being whether "ota-wipe=yes" is set.
Doug Zongkereef39442009-04-02 12:14:19 -070062
Tao Bao30df8b42018-04-23 15:32:53 -070063 --wipe_user_data
64 Generate an OTA package that will wipe the user data partition when
65 installed.
66
Yifan Hong50e79542018-11-08 17:44:12 -080067 --retrofit_dynamic_partitions
68 Generates an OTA package that updates a device to support dynamic
69 partitions (default False). This flag is implied when generating
70 an incremental OTA where the base build does not support dynamic
71 partitions but the target build does. For A/B, when this flag is set,
72 --skip_postinstall is implied.
73
xunchangabfa2652019-02-19 16:27:10 -080074 --skip_compatibility_check
Yifan Hong9276cf02019-08-21 16:37:04 -070075 Skip checking compatibility of the input target files package.
xunchangabfa2652019-02-19 16:27:10 -080076
xunchang1cfe2512019-02-19 14:14:48 -080077 --output_metadata_path
78 Write a copy of the metadata to a separate file. Therefore, users can
79 read the post build fingerprint without extracting the OTA package.
80
Tao Bao30df8b42018-04-23 15:32:53 -070081Non-A/B OTA specific options
82
83 -b (--binary) <file>
84 Use the given binary as the update-binary in the output package, instead
85 of the binary in the build's target_files. Use for development only.
86
87 --block
88 Generate a block-based OTA for non-A/B device. We have deprecated the
89 support for file-based OTA since O. Block-based OTA will be used by
90 default for all non-A/B devices. Keeping this flag here to not break
91 existing callers.
92
93 -e (--extra_script) <file>
94 Insert the contents of file at the end of the update script.
Tao Bao43078aa2015-04-21 14:32:35 -070095
leozwangaa6c1a12015-08-14 10:57:58 -070096 --full_bootloader
97 Similar to --full_radio. When generating an incremental OTA, always
98 include a full copy of bootloader image.
99
Tao Bao30df8b42018-04-23 15:32:53 -0700100 --full_radio
101 When generating an incremental OTA, always include a full copy of radio
102 image. This option is only meaningful when -i is specified, because a full
103 radio is always included in a full OTA if applicable.
Michael Runge63f01de2014-10-28 19:24:19 -0700104
Tao Bao30df8b42018-04-23 15:32:53 -0700105 --log_diff <file>
106 Generate a log file that shows the differences in the source and target
107 builds for an incremental package. This option is only meaningful when -i
108 is specified.
109
110 -o (--oem_settings) <main_file[,additional_files...]>
Alain Vongsouvanh7f804ba2017-02-16 13:06:55 -0800111 Comma seperated list of files used to specify the expected OEM-specific
Tao Bao481bab82017-12-21 11:23:09 -0800112 properties on the OEM partition of the intended device. Multiple expected
113 values can be used by providing multiple files. Only the first dict will
114 be used to compute fingerprint, while the rest will be used to assert
115 OEM-specific properties.
Alain Vongsouvanh7f804ba2017-02-16 13:06:55 -0800116
Tao Bao8608cde2016-02-25 19:49:55 -0800117 --oem_no_mount
Tao Bao30df8b42018-04-23 15:32:53 -0700118 For devices with OEM-specific properties but without an OEM partition, do
119 not mount the OEM partition in the updater-script. This should be very
120 rarely used, since it's expected to have a dedicated OEM partition for
121 OEM-specific properties. Only meaningful when -o is specified.
Tao Bao8608cde2016-02-25 19:49:55 -0800122
Tao Bao30df8b42018-04-23 15:32:53 -0700123 --stash_threshold <float>
124 Specify the threshold that will be used to compute the maximum allowed
125 stash size (defaults to 0.8).
Doug Zongkerdbfaae52009-04-21 17:12:54 -0700126
Tao Bao30df8b42018-04-23 15:32:53 -0700127 -t (--worker_threads) <int>
128 Specify the number of worker-threads that will be used when generating
129 patches for incremental updates (defaults to 3).
Tao Bao3e6161a2017-02-28 11:48:48 -0800130
Tao Bao30df8b42018-04-23 15:32:53 -0700131 --verify
132 Verify the checksums of the updated system and vendor (if any) partitions.
133 Non-A/B incremental OTAs only.
Doug Zongker1c390a22009-05-14 19:06:36 -0700134
Doug Zongker9b23f2c2013-11-25 14:44:12 -0800135 -2 (--two_step)
Tao Bao30df8b42018-04-23 15:32:53 -0700136 Generate a 'two-step' OTA package, where recovery is updated first, so
137 that any changes made to the system partition are done using the new
138 recovery (new kernel, etc.).
139
140A/B OTA specific options
Doug Zongker9b23f2c2013-11-25 14:44:12 -0800141
Tianjie Xu1b079832019-08-28 12:19:23 -0700142 --disable_fec_computation
143 Disable the on device FEC data computation for incremental updates.
144
Tao Baof7140c02018-01-30 17:09:24 -0800145 --include_secondary
146 Additionally include the payload for secondary slot images (default:
147 False). Only meaningful when generating A/B OTAs.
148
149 By default, an A/B OTA package doesn't contain the images for the
150 secondary slot (e.g. system_other.img). Specifying this flag allows
151 generating a separate payload that will install secondary slot images.
152
153 Such a package needs to be applied in a two-stage manner, with a reboot
154 in-between. During the first stage, the updater applies the primary
155 payload only. Upon finishing, it reboots the device into the newly updated
156 slot. It then continues to install the secondary payload to the inactive
157 slot, but without switching the active slot at the end (needs the matching
158 support in update_engine, i.e. SWITCH_SLOT_ON_REBOOT flag).
159
160 Due to the special install procedure, the secondary payload will be always
161 generated as a full payload.
162
Tao Baodea0f8b2016-06-20 17:55:06 -0700163 --payload_signer <signer>
164 Specify the signer when signing the payload and metadata for A/B OTAs.
165 By default (i.e. without this flag), it calls 'openssl pkeyutl' to sign
166 with the package private key. If the private key cannot be accessed
167 directly, a payload signer that knows how to do that should be specified.
168 The signer will be supplied with "-inkey <path_to_key>",
169 "-in <input_file>" and "-out <output_file>" parameters.
Baligh Uddin2abbbd02016-06-22 12:14:16 -0700170
171 --payload_signer_args <args>
172 Specify the arguments needed for payload signer.
Tao Bao15a146a2018-02-21 16:06:59 -0800173
xunchang376cc7c2019-04-08 23:04:58 -0700174 --payload_signer_key_size <key_size>
175 Specify the key size in bytes of the payload signer.
176
Tao Bao15a146a2018-02-21 16:06:59 -0800177 --skip_postinstall
178 Skip the postinstall hooks when generating an A/B OTA package (default:
179 False). Note that this discards ALL the hooks, including non-optional
180 ones. Should only be used if caller knows it's safe to do so (e.g. all the
181 postinstall work is to dexopt apps and a data wipe will happen immediately
182 after). Only meaningful when generating A/B OTAs.
Doug Zongkereef39442009-04-02 12:14:19 -0700183"""
184
Tao Bao89fbb0f2017-01-10 10:47:58 -0800185from __future__ import print_function
186
Tianjie Xuf67dd802019-05-20 17:50:36 -0700187import collections
Tao Bao32fcdab2018-10-12 10:30:39 -0700188import logging
Doug Zongkerfc44a512014-08-26 13:10:25 -0700189import multiprocessing
Tao Bao2dd1c482017-02-03 16:49:39 -0800190import os.path
Baligh Uddin2abbbd02016-06-22 12:14:16 -0700191import shlex
Tao Bao15a146a2018-02-21 16:06:59 -0800192import shutil
Tao Bao85f16982018-03-08 16:28:33 -0800193import struct
Tao Bao481bab82017-12-21 11:23:09 -0800194import sys
Doug Zongkereef39442009-04-02 12:14:19 -0700195import zipfile
196
Yifan Hong9276cf02019-08-21 16:37:04 -0700197import check_target_files_vintf
Doug Zongkereef39442009-04-02 12:14:19 -0700198import common
Doug Zongkerc494d7c2009-06-18 08:43:44 -0700199import edify_generator
Tianjie Xu67c7cbb2018-08-30 00:32:07 -0700200import verity_utils
Doug Zongkereef39442009-04-02 12:14:19 -0700201
Tao Bao481bab82017-12-21 11:23:09 -0800202if sys.hexversion < 0x02070000:
203 print("Python 2.7 or newer is required.", file=sys.stderr)
204 sys.exit(1)
205
Tao Bao32fcdab2018-10-12 10:30:39 -0700206logger = logging.getLogger(__name__)
Tao Bao481bab82017-12-21 11:23:09 -0800207
Doug Zongkereef39442009-04-02 12:14:19 -0700208OPTIONS = common.OPTIONS
Doug Zongkerafb32ea2011-09-22 10:28:04 -0700209OPTIONS.package_key = None
Doug Zongkereef39442009-04-02 12:14:19 -0700210OPTIONS.incremental_source = None
Michael Runge63f01de2014-10-28 19:24:19 -0700211OPTIONS.verify = False
Doug Zongkereef39442009-04-02 12:14:19 -0700212OPTIONS.patch_threshold = 0.95
Doug Zongkerdbfaae52009-04-21 17:12:54 -0700213OPTIONS.wipe_user_data = False
Tao Bao5d182562016-02-23 11:38:39 -0800214OPTIONS.downgrade = False
Doug Zongker1c390a22009-05-14 19:06:36 -0700215OPTIONS.extra_script = None
Doug Zongkerfc44a512014-08-26 13:10:25 -0700216OPTIONS.worker_threads = multiprocessing.cpu_count() // 2
217if OPTIONS.worker_threads == 0:
218 OPTIONS.worker_threads = 1
Doug Zongker9b23f2c2013-11-25 14:44:12 -0800219OPTIONS.two_step = False
Tao Baof7140c02018-01-30 17:09:24 -0800220OPTIONS.include_secondary = False
Takeshi Kanemotoe153b342013-11-14 17:20:50 +0900221OPTIONS.no_signing = False
Tao Bao457cbf62017-03-06 09:56:01 -0800222OPTIONS.block_based = True
Doug Zongker25568482014-03-03 10:21:27 -0800223OPTIONS.updater_binary = None
Michael Runge6e836112014-04-15 17:40:21 -0700224OPTIONS.oem_source = None
Tao Bao8608cde2016-02-25 19:49:55 -0800225OPTIONS.oem_no_mount = False
Tao Bao43078aa2015-04-21 14:32:35 -0700226OPTIONS.full_radio = False
leozwangaa6c1a12015-08-14 10:57:58 -0700227OPTIONS.full_bootloader = False
Tao Baod47d8e12015-05-21 14:09:49 -0700228# Stash size cannot exceed cache_size * threshold.
229OPTIONS.cache_size = None
230OPTIONS.stash_threshold = 0.8
Tao Baod62c6032015-11-30 09:40:20 -0800231OPTIONS.log_diff = None
Tao Baodea0f8b2016-06-20 17:55:06 -0700232OPTIONS.payload_signer = None
Baligh Uddin2abbbd02016-06-22 12:14:16 -0700233OPTIONS.payload_signer_args = []
xunchang376cc7c2019-04-08 23:04:58 -0700234OPTIONS.payload_signer_key_size = None
Tao Bao5f8ff932017-03-21 22:35:00 -0700235OPTIONS.extracted_input = None
Christian Oderf63e2cd2017-05-01 22:30:15 +0200236OPTIONS.key_passwords = []
Tao Bao15a146a2018-02-21 16:06:59 -0800237OPTIONS.skip_postinstall = False
Yifan Hong50e79542018-11-08 17:44:12 -0800238OPTIONS.retrofit_dynamic_partitions = False
xunchangabfa2652019-02-19 16:27:10 -0800239OPTIONS.skip_compatibility_check = False
xunchang1cfe2512019-02-19 14:14:48 -0800240OPTIONS.output_metadata_path = None
Tianjie Xu1b079832019-08-28 12:19:23 -0700241OPTIONS.disable_fec_computation = False
Tao Bao15a146a2018-02-21 16:06:59 -0800242
Tao Bao8dcf7382015-05-21 14:09:49 -0700243
Tao Bao2dd1c482017-02-03 16:49:39 -0800244METADATA_NAME = 'META-INF/com/android/metadata'
Tao Bao15a146a2018-02-21 16:06:59 -0800245POSTINSTALL_CONFIG = 'META/postinstall_config.txt'
Yifan Hong50e79542018-11-08 17:44:12 -0800246DYNAMIC_PARTITION_INFO = 'META/dynamic_partitions_info.txt'
Yifan Hongb433eba2019-03-06 12:42:53 -0800247AB_PARTITIONS = 'META/ab_partitions.txt'
Tao Bao04808502019-07-25 23:11:41 -0700248UNZIP_PATTERN = ['IMAGES/*', 'META/*', 'OTA/*', 'RADIO/*']
Tao Baof0c4aa22018-04-30 20:29:30 -0700249# Files to be unzipped for target diffing purpose.
250TARGET_DIFFING_UNZIP_PATTERN = ['BOOT', 'RECOVERY', 'SYSTEM/*', 'VENDOR/*',
251 'PRODUCT/*', 'SYSTEM_EXT/*', 'ODM/*']
Yifan Hongb433eba2019-03-06 12:42:53 -0800252RETROFIT_DAP_UNZIP_PATTERN = ['OTA/super_*.img', AB_PARTITIONS]
Tianjie Xu1c808002019-09-11 00:29:26 -0700253SECONDARY_IMAGES_SKIP_PARTITIONS = ['odm', 'product', 'system_ext', 'vendor']
Tao Bao6b0b2f92017-03-05 11:38:11 -0800254
Tao Bao2dd1c482017-02-03 16:49:39 -0800255
Tao Bao481bab82017-12-21 11:23:09 -0800256class BuildInfo(object):
257 """A class that holds the information for a given build.
258
259 This class wraps up the property querying for a given source or target build.
260 It abstracts away the logic of handling OEM-specific properties, and caches
261 the commonly used properties such as fingerprint.
262
263 There are two types of info dicts: a) build-time info dict, which is generated
264 at build time (i.e. included in a target_files zip); b) OEM info dict that is
265 specified at package generation time (via command line argument
266 '--oem_settings'). If a build doesn't use OEM-specific properties (i.e. not
267 having "oem_fingerprint_properties" in build-time info dict), all the queries
268 would be answered based on build-time info dict only. Otherwise if using
269 OEM-specific properties, some of them will be calculated from two info dicts.
270
271 Users can query properties similarly as using a dict() (e.g. info['fstab']),
272 or to query build properties via GetBuildProp() or GetVendorBuildProp().
273
274 Attributes:
275 info_dict: The build-time info dict.
276 is_ab: Whether it's a build that uses A/B OTA.
277 oem_dicts: A list of OEM dicts.
278 oem_props: A list of OEM properties that should be read from OEM dicts; None
279 if the build doesn't use any OEM-specific property.
280 fingerprint: The fingerprint of the build, which would be calculated based
281 on OEM properties if applicable.
282 device: The device name, which could come from OEM dicts if applicable.
283 """
284
Steven Laver9e73e822019-01-29 20:20:08 -0800285 _RO_PRODUCT_RESOLVE_PROPS = ["ro.product.brand", "ro.product.device",
286 "ro.product.manufacturer", "ro.product.model",
287 "ro.product.name"]
Justin Yun6151e3f2019-06-25 15:58:13 +0900288 _RO_PRODUCT_PROPS_DEFAULT_SOURCE_ORDER = ["product", "odm", "vendor",
289 "system_ext", "system"]
Steven Laver9e73e822019-01-29 20:20:08 -0800290
Tao Bao481bab82017-12-21 11:23:09 -0800291 def __init__(self, info_dict, oem_dicts):
292 """Initializes a BuildInfo instance with the given dicts.
293
Tao Bao667c7532018-07-06 10:13:59 -0700294 Note that it only wraps up the given dicts, without making copies.
295
Tao Bao481bab82017-12-21 11:23:09 -0800296 Arguments:
297 info_dict: The build-time info dict.
298 oem_dicts: A list of OEM dicts (which is parsed from --oem_settings). Note
299 that it always uses the first dict to calculate the fingerprint or the
300 device name. The rest would be used for asserting OEM properties only
Tao Bao667c7532018-07-06 10:13:59 -0700301 (e.g. one package can be installed on one of these devices).
Tao Bao481bab82017-12-21 11:23:09 -0800302 """
303 self.info_dict = info_dict
304 self.oem_dicts = oem_dicts
305
306 self._is_ab = info_dict.get("ab_update") == "true"
307 self._oem_props = info_dict.get("oem_fingerprint_properties")
308
309 if self._oem_props:
310 assert oem_dicts, "OEM source required for this build"
311
312 # These two should be computed only after setting self._oem_props.
313 self._device = self.GetOemProperty("ro.product.device")
314 self._fingerprint = self.CalculateFingerprint()
315
316 @property
317 def is_ab(self):
318 return self._is_ab
319
320 @property
321 def device(self):
322 return self._device
323
324 @property
325 def fingerprint(self):
326 return self._fingerprint
327
328 @property
Tao Baoea6cbd02018-09-05 13:06:37 -0700329 def vendor_fingerprint(self):
Yifan Hong51d37562019-04-23 17:06:46 -0700330 return self._fingerprint_of("vendor")
331
332 @property
333 def product_fingerprint(self):
334 return self._fingerprint_of("product")
335
336 @property
337 def odm_fingerprint(self):
338 return self._fingerprint_of("odm")
339
340 def _fingerprint_of(self, partition):
341 if partition + ".build.prop" not in self.info_dict:
Tao Baoea6cbd02018-09-05 13:06:37 -0700342 return None
Yifan Hong51d37562019-04-23 17:06:46 -0700343 build_prop = self.info_dict[partition + ".build.prop"]
344 if "ro." + partition + ".build.fingerprint" in build_prop:
345 return build_prop["ro." + partition + ".build.fingerprint"]
346 if "ro." + partition + ".build.thumbprint" in build_prop:
347 return build_prop["ro." + partition + ".build.thumbprint"]
Tao Baoea6cbd02018-09-05 13:06:37 -0700348 return None
349
350 @property
Tao Bao481bab82017-12-21 11:23:09 -0800351 def oem_props(self):
352 return self._oem_props
353
354 def __getitem__(self, key):
355 return self.info_dict[key]
356
Tao Bao667c7532018-07-06 10:13:59 -0700357 def __setitem__(self, key, value):
358 self.info_dict[key] = value
359
Tao Bao481bab82017-12-21 11:23:09 -0800360 def get(self, key, default=None):
361 return self.info_dict.get(key, default)
362
Tao Bao667c7532018-07-06 10:13:59 -0700363 def items(self):
364 return self.info_dict.items()
365
Tao Bao481bab82017-12-21 11:23:09 -0800366 def GetBuildProp(self, prop):
367 """Returns the inquired build property."""
Steven Laver9e73e822019-01-29 20:20:08 -0800368 if prop in BuildInfo._RO_PRODUCT_RESOLVE_PROPS:
369 return self._ResolveRoProductBuildProp(prop)
370
Tao Bao481bab82017-12-21 11:23:09 -0800371 try:
372 return self.info_dict.get("build.prop", {})[prop]
373 except KeyError:
374 raise common.ExternalError("couldn't find %s in build.prop" % (prop,))
375
Steven Laver9e73e822019-01-29 20:20:08 -0800376 def _ResolveRoProductBuildProp(self, prop):
377 """Resolves the inquired ro.product.* build property"""
378 prop_val = self.info_dict.get("build.prop", {}).get(prop)
379 if prop_val:
380 return prop_val
381
382 source_order_val = self.info_dict.get("build.prop", {}).get(
Tao Bao59cf0c52019-06-25 10:04:24 -0700383 "ro.product.property_source_order")
Steven Laver9e73e822019-01-29 20:20:08 -0800384 if source_order_val:
385 source_order = source_order_val.split(",")
386 else:
387 source_order = BuildInfo._RO_PRODUCT_PROPS_DEFAULT_SOURCE_ORDER
388
389 # Check that all sources in ro.product.property_source_order are valid
390 if any([x not in BuildInfo._RO_PRODUCT_PROPS_DEFAULT_SOURCE_ORDER
391 for x in source_order]):
392 raise common.ExternalError(
Tao Bao59cf0c52019-06-25 10:04:24 -0700393 "Invalid ro.product.property_source_order '{}'".format(source_order))
Steven Laver9e73e822019-01-29 20:20:08 -0800394
395 for source in source_order:
Tao Bao59cf0c52019-06-25 10:04:24 -0700396 source_prop = prop.replace(
397 "ro.product", "ro.product.{}".format(source), 1)
398 prop_val = self.info_dict.get(
399 "{}.build.prop".format(source), {}).get(source_prop)
Steven Laver9e73e822019-01-29 20:20:08 -0800400 if prop_val:
401 return prop_val
402
403 raise common.ExternalError("couldn't resolve {}".format(prop))
404
Tao Bao481bab82017-12-21 11:23:09 -0800405 def GetVendorBuildProp(self, prop):
406 """Returns the inquired vendor build property."""
407 try:
408 return self.info_dict.get("vendor.build.prop", {})[prop]
409 except KeyError:
410 raise common.ExternalError(
411 "couldn't find %s in vendor.build.prop" % (prop,))
412
413 def GetOemProperty(self, key):
414 if self.oem_props is not None and key in self.oem_props:
415 return self.oem_dicts[0][key]
416 return self.GetBuildProp(key)
417
418 def CalculateFingerprint(self):
419 if self.oem_props is None:
Steven Laver9e73e822019-01-29 20:20:08 -0800420 try:
421 return self.GetBuildProp("ro.build.fingerprint")
422 except common.ExternalError:
423 return "{}/{}/{}:{}/{}/{}:{}/{}".format(
Tao Bao59cf0c52019-06-25 10:04:24 -0700424 self.GetBuildProp("ro.product.brand"),
425 self.GetBuildProp("ro.product.name"),
426 self.GetBuildProp("ro.product.device"),
427 self.GetBuildProp("ro.build.version.release"),
428 self.GetBuildProp("ro.build.id"),
429 self.GetBuildProp("ro.build.version.incremental"),
430 self.GetBuildProp("ro.build.type"),
431 self.GetBuildProp("ro.build.tags"))
Tao Bao481bab82017-12-21 11:23:09 -0800432 return "%s/%s/%s:%s" % (
433 self.GetOemProperty("ro.product.brand"),
434 self.GetOemProperty("ro.product.name"),
435 self.GetOemProperty("ro.product.device"),
436 self.GetBuildProp("ro.build.thumbprint"))
437
438 def WriteMountOemScript(self, script):
439 assert self.oem_props is not None
440 recovery_mount_options = self.info_dict.get("recovery_mount_options")
441 script.Mount("/oem", recovery_mount_options)
442
443 def WriteDeviceAssertions(self, script, oem_no_mount):
444 # Read the property directly if not using OEM properties.
445 if not self.oem_props:
446 script.AssertDevice(self.device)
447 return
448
449 # Otherwise assert OEM properties.
450 if not self.oem_dicts:
451 raise common.ExternalError(
452 "No OEM file provided to answer expected assertions")
453
454 for prop in self.oem_props.split():
455 values = []
456 for oem_dict in self.oem_dicts:
457 if prop in oem_dict:
458 values.append(oem_dict[prop])
459 if not values:
460 raise common.ExternalError(
461 "The OEM file is missing the property %s" % (prop,))
462 script.AssertOemProperty(prop, values, oem_no_mount)
463
464
Tao Baofabe0832018-01-17 15:52:28 -0800465class PayloadSigner(object):
466 """A class that wraps the payload signing works.
467
468 When generating a Payload, hashes of the payload and metadata files will be
469 signed with the device key, either by calling an external payload signer or
470 by calling openssl with the package key. This class provides a unified
471 interface, so that callers can just call PayloadSigner.Sign().
472
473 If an external payload signer has been specified (OPTIONS.payload_signer), it
474 calls the signer with the provided args (OPTIONS.payload_signer_args). Note
475 that the signing key should be provided as part of the payload_signer_args.
476 Otherwise without an external signer, it uses the package key
477 (OPTIONS.package_key) and calls openssl for the signing works.
478 """
479
480 def __init__(self):
481 if OPTIONS.payload_signer is None:
482 # Prepare the payload signing key.
483 private_key = OPTIONS.package_key + OPTIONS.private_key_suffix
484 pw = OPTIONS.key_passwords[OPTIONS.package_key]
485
486 cmd = ["openssl", "pkcs8", "-in", private_key, "-inform", "DER"]
487 cmd.extend(["-passin", "pass:" + pw] if pw else ["-nocrypt"])
488 signing_key = common.MakeTempFile(prefix="key-", suffix=".key")
489 cmd.extend(["-out", signing_key])
Tao Baobec89c12018-10-15 11:53:28 -0700490 common.RunAndCheckOutput(cmd, verbose=False)
Tao Baofabe0832018-01-17 15:52:28 -0800491
492 self.signer = "openssl"
493 self.signer_args = ["pkeyutl", "-sign", "-inkey", signing_key,
494 "-pkeyopt", "digest:sha256"]
xunchang376cc7c2019-04-08 23:04:58 -0700495 self.key_size = self._GetKeySizeInBytes(signing_key)
Tao Baofabe0832018-01-17 15:52:28 -0800496 else:
497 self.signer = OPTIONS.payload_signer
498 self.signer_args = OPTIONS.payload_signer_args
xunchang376cc7c2019-04-08 23:04:58 -0700499 if OPTIONS.payload_signer_key_size:
500 self.key_size = int(OPTIONS.payload_signer_key_size)
501 assert self.key_size == 256 or self.key_size == 512, \
502 "Unsupported key size {}".format(OPTIONS.payload_signer_key_size)
503 else:
504 self.key_size = 256
505
506 @staticmethod
507 def _GetKeySizeInBytes(signing_key):
508 modulus_file = common.MakeTempFile(prefix="modulus-")
509 cmd = ["openssl", "rsa", "-inform", "PEM", "-in", signing_key, "-modulus",
510 "-noout", "-out", modulus_file]
511 common.RunAndCheckOutput(cmd, verbose=False)
512
513 with open(modulus_file) as f:
514 modulus_string = f.read()
515 # The modulus string has the format "Modulus=$data", where $data is the
516 # concatenation of hex dump of the modulus.
517 MODULUS_PREFIX = "Modulus="
518 assert modulus_string.startswith(MODULUS_PREFIX)
519 modulus_string = modulus_string[len(MODULUS_PREFIX):]
Tao Bao59cf0c52019-06-25 10:04:24 -0700520 key_size = len(modulus_string) // 2
xunchang376cc7c2019-04-08 23:04:58 -0700521 assert key_size == 256 or key_size == 512, \
522 "Unsupported key size {}".format(key_size)
523 return key_size
Tao Baofabe0832018-01-17 15:52:28 -0800524
525 def Sign(self, in_file):
526 """Signs the given input file. Returns the output filename."""
527 out_file = common.MakeTempFile(prefix="signed-", suffix=".bin")
528 cmd = [self.signer] + self.signer_args + ['-in', in_file, '-out', out_file]
Tao Bao718faed2019-08-02 13:24:19 -0700529 common.RunAndCheckOutput(cmd)
Tao Baofabe0832018-01-17 15:52:28 -0800530 return out_file
531
532
Tao Bao40b18822018-01-30 18:19:04 -0800533class Payload(object):
534 """Manages the creation and the signing of an A/B OTA Payload."""
535
536 PAYLOAD_BIN = 'payload.bin'
537 PAYLOAD_PROPERTIES_TXT = 'payload_properties.txt'
Tao Baof7140c02018-01-30 17:09:24 -0800538 SECONDARY_PAYLOAD_BIN = 'secondary/payload.bin'
539 SECONDARY_PAYLOAD_PROPERTIES_TXT = 'secondary/payload_properties.txt'
Tao Bao40b18822018-01-30 18:19:04 -0800540
Tao Bao667ff572018-02-10 00:02:40 -0800541 def __init__(self, secondary=False):
542 """Initializes a Payload instance.
543
544 Args:
545 secondary: Whether it's generating a secondary payload (default: False).
546 """
Tao Bao40b18822018-01-30 18:19:04 -0800547 self.payload_file = None
548 self.payload_properties = None
Tao Bao667ff572018-02-10 00:02:40 -0800549 self.secondary = secondary
Tao Bao40b18822018-01-30 18:19:04 -0800550
Tao Baof0c4aa22018-04-30 20:29:30 -0700551 def _Run(self, cmd): # pylint: disable=no-self-use
Tao Bao718faed2019-08-02 13:24:19 -0700552 # Don't pipe (buffer) the output if verbose is set. Let
553 # brillo_update_payload write to stdout/stderr directly, so its progress can
554 # be monitored.
555 if OPTIONS.verbose:
556 common.RunAndCheckOutput(cmd, stdout=None, stderr=None)
557 else:
558 common.RunAndCheckOutput(cmd)
559
Tao Bao40b18822018-01-30 18:19:04 -0800560 def Generate(self, target_file, source_file=None, additional_args=None):
561 """Generates a payload from the given target-files zip(s).
562
563 Args:
564 target_file: The filename of the target build target-files zip.
565 source_file: The filename of the source build target-files zip; or None if
566 generating a full OTA.
567 additional_args: A list of additional args that should be passed to
568 brillo_update_payload script; or None.
569 """
570 if additional_args is None:
571 additional_args = []
572
573 payload_file = common.MakeTempFile(prefix="payload-", suffix=".bin")
574 cmd = ["brillo_update_payload", "generate",
575 "--payload", payload_file,
576 "--target_image", target_file]
577 if source_file is not None:
578 cmd.extend(["--source_image", source_file])
Tianjie Xu1b079832019-08-28 12:19:23 -0700579 if OPTIONS.disable_fec_computation:
580 cmd.extend(["--disable_fec_computation", "true"])
Tao Bao40b18822018-01-30 18:19:04 -0800581 cmd.extend(additional_args)
Tao Bao718faed2019-08-02 13:24:19 -0700582 self._Run(cmd)
Tao Bao40b18822018-01-30 18:19:04 -0800583
584 self.payload_file = payload_file
585 self.payload_properties = None
586
587 def Sign(self, payload_signer):
588 """Generates and signs the hashes of the payload and metadata.
589
590 Args:
591 payload_signer: A PayloadSigner() instance that serves the signing work.
592
593 Raises:
594 AssertionError: On any failure when calling brillo_update_payload script.
595 """
596 assert isinstance(payload_signer, PayloadSigner)
597
598 # 1. Generate hashes of the payload and metadata files.
599 payload_sig_file = common.MakeTempFile(prefix="sig-", suffix=".bin")
600 metadata_sig_file = common.MakeTempFile(prefix="sig-", suffix=".bin")
601 cmd = ["brillo_update_payload", "hash",
602 "--unsigned_payload", self.payload_file,
xunchang376cc7c2019-04-08 23:04:58 -0700603 "--signature_size", str(payload_signer.key_size),
Tao Bao40b18822018-01-30 18:19:04 -0800604 "--metadata_hash_file", metadata_sig_file,
605 "--payload_hash_file", payload_sig_file]
Tao Bao718faed2019-08-02 13:24:19 -0700606 self._Run(cmd)
Tao Bao40b18822018-01-30 18:19:04 -0800607
608 # 2. Sign the hashes.
609 signed_payload_sig_file = payload_signer.Sign(payload_sig_file)
610 signed_metadata_sig_file = payload_signer.Sign(metadata_sig_file)
611
612 # 3. Insert the signatures back into the payload file.
613 signed_payload_file = common.MakeTempFile(prefix="signed-payload-",
614 suffix=".bin")
615 cmd = ["brillo_update_payload", "sign",
616 "--unsigned_payload", self.payload_file,
617 "--payload", signed_payload_file,
xunchang376cc7c2019-04-08 23:04:58 -0700618 "--signature_size", str(payload_signer.key_size),
Tao Bao40b18822018-01-30 18:19:04 -0800619 "--metadata_signature_file", signed_metadata_sig_file,
620 "--payload_signature_file", signed_payload_sig_file]
Tao Bao718faed2019-08-02 13:24:19 -0700621 self._Run(cmd)
Tao Bao40b18822018-01-30 18:19:04 -0800622
623 # 4. Dump the signed payload properties.
624 properties_file = common.MakeTempFile(prefix="payload-properties-",
625 suffix=".txt")
626 cmd = ["brillo_update_payload", "properties",
627 "--payload", signed_payload_file,
628 "--properties_file", properties_file]
Tao Bao718faed2019-08-02 13:24:19 -0700629 self._Run(cmd)
Tao Bao40b18822018-01-30 18:19:04 -0800630
Tao Bao667ff572018-02-10 00:02:40 -0800631 if self.secondary:
632 with open(properties_file, "a") as f:
633 f.write("SWITCH_SLOT_ON_REBOOT=0\n")
634
Tao Bao40b18822018-01-30 18:19:04 -0800635 if OPTIONS.wipe_user_data:
636 with open(properties_file, "a") as f:
637 f.write("POWERWASH=1\n")
638
639 self.payload_file = signed_payload_file
640 self.payload_properties = properties_file
641
Tao Bao667ff572018-02-10 00:02:40 -0800642 def WriteToZip(self, output_zip):
Tao Bao40b18822018-01-30 18:19:04 -0800643 """Writes the payload to the given zip.
644
645 Args:
646 output_zip: The output ZipFile instance.
647 """
648 assert self.payload_file is not None
649 assert self.payload_properties is not None
650
Tao Bao667ff572018-02-10 00:02:40 -0800651 if self.secondary:
Tao Baof7140c02018-01-30 17:09:24 -0800652 payload_arcname = Payload.SECONDARY_PAYLOAD_BIN
653 payload_properties_arcname = Payload.SECONDARY_PAYLOAD_PROPERTIES_TXT
654 else:
655 payload_arcname = Payload.PAYLOAD_BIN
656 payload_properties_arcname = Payload.PAYLOAD_PROPERTIES_TXT
657
Tao Bao40b18822018-01-30 18:19:04 -0800658 # Add the signed payload file and properties into the zip. In order to
659 # support streaming, we pack them as ZIP_STORED. So these entries can be
660 # read directly with the offset and length pairs.
Tao Baof7140c02018-01-30 17:09:24 -0800661 common.ZipWrite(output_zip, self.payload_file, arcname=payload_arcname,
Tao Bao40b18822018-01-30 18:19:04 -0800662 compress_type=zipfile.ZIP_STORED)
663 common.ZipWrite(output_zip, self.payload_properties,
Tao Baof7140c02018-01-30 17:09:24 -0800664 arcname=payload_properties_arcname,
Tao Bao40b18822018-01-30 18:19:04 -0800665 compress_type=zipfile.ZIP_STORED)
666
667
Doug Zongkereef39442009-04-02 12:14:19 -0700668def SignOutput(temp_zip_name, output_zip_name):
Christian Oderf63e2cd2017-05-01 22:30:15 +0200669 pw = OPTIONS.key_passwords[OPTIONS.package_key]
Doug Zongkereef39442009-04-02 12:14:19 -0700670
Doug Zongker951495f2009-08-14 12:44:19 -0700671 common.SignFile(temp_zip_name, output_zip_name, OPTIONS.package_key, pw,
672 whole_file=True)
Doug Zongkereef39442009-04-02 12:14:19 -0700673
674
Tao Bao481bab82017-12-21 11:23:09 -0800675def _LoadOemDicts(oem_source):
Alain Vongsouvanh7f804ba2017-02-16 13:06:55 -0800676 """Returns the list of loaded OEM properties dict."""
Tao Bao481bab82017-12-21 11:23:09 -0800677 if not oem_source:
678 return None
679
Alain Vongsouvanh7f804ba2017-02-16 13:06:55 -0800680 oem_dicts = []
Tao Bao481bab82017-12-21 11:23:09 -0800681 for oem_file in oem_source:
682 with open(oem_file) as fp:
683 oem_dicts.append(common.LoadDictionaryFromLines(fp.readlines()))
Alain Vongsouvanh7f804ba2017-02-16 13:06:55 -0800684 return oem_dicts
Doug Zongkereef39442009-04-02 12:14:19 -0700685
Doug Zongkereef39442009-04-02 12:14:19 -0700686
Tao Baod42e97e2016-11-30 12:11:57 -0800687def _WriteRecoveryImageToBoot(script, output_zip):
688 """Find and write recovery image to /boot in two-step OTA.
689
690 In two-step OTAs, we write recovery image to /boot as the first step so that
691 we can reboot to there and install a new recovery image to /recovery.
692 A special "recovery-two-step.img" will be preferred, which encodes the correct
693 path of "/boot". Otherwise the device may show "device is corrupt" message
694 when booting into /boot.
695
696 Fall back to using the regular recovery.img if the two-step recovery image
697 doesn't exist. Note that rebuilding the special image at this point may be
698 infeasible, because we don't have the desired boot signer and keys when
699 calling ota_from_target_files.py.
700 """
701
702 recovery_two_step_img_name = "recovery-two-step.img"
703 recovery_two_step_img_path = os.path.join(
Tao Bao04808502019-07-25 23:11:41 -0700704 OPTIONS.input_tmp, "OTA", recovery_two_step_img_name)
Tao Baod42e97e2016-11-30 12:11:57 -0800705 if os.path.exists(recovery_two_step_img_path):
Tao Bao04808502019-07-25 23:11:41 -0700706 common.ZipWrite(
707 output_zip,
708 recovery_two_step_img_path,
709 arcname=recovery_two_step_img_name)
Tao Bao32fcdab2018-10-12 10:30:39 -0700710 logger.info(
711 "two-step package: using %s in stage 1/3", recovery_two_step_img_name)
Tao Baod42e97e2016-11-30 12:11:57 -0800712 script.WriteRawImage("/boot", recovery_two_step_img_name)
713 else:
Tao Bao32fcdab2018-10-12 10:30:39 -0700714 logger.info("two-step package: using recovery.img in stage 1/3")
Tao Baod42e97e2016-11-30 12:11:57 -0800715 # The "recovery.img" entry has been written into package earlier.
716 script.WriteRawImage("/boot", "recovery.img")
717
718
Doug Zongkerc9253822014-02-04 12:17:58 -0800719def HasRecoveryPatch(target_files_zip):
Tao Baof2cffbd2015-07-22 12:33:18 -0700720 namelist = [name for name in target_files_zip.namelist()]
721 return ("SYSTEM/recovery-from-boot.p" in namelist or
722 "SYSTEM/etc/recovery.img" in namelist)
Doug Zongker73ef8252009-07-23 15:12:53 -0700723
Tao Bao457cbf62017-03-06 09:56:01 -0800724
Yifan Hong51d37562019-04-23 17:06:46 -0700725def HasPartition(target_files_zip, partition):
Doug Zongkerc8b4e842014-06-16 15:16:31 -0700726 try:
Yifan Hong51d37562019-04-23 17:06:46 -0700727 target_files_zip.getinfo(partition.upper() + "/")
Doug Zongkerc8b4e842014-06-16 15:16:31 -0700728 return True
729 except KeyError:
730 return False
731
Tao Bao457cbf62017-03-06 09:56:01 -0800732
Yifan Hong9276cf02019-08-21 16:37:04 -0700733def HasTrebleEnabled(target_files, target_info):
734 def HasVendorPartition(target_files):
735 if os.path.isdir(target_files):
736 return os.path.isdir(os.path.join(target_files, "VENDOR"))
737 if zipfile.is_zipfile(target_files):
738 return HasPartition(zipfile.ZipFile(target_files), "vendor")
739 raise ValueError("Unknown target_files argument")
Yifan Hong51d37562019-04-23 17:06:46 -0700740
Yifan Hong9276cf02019-08-21 16:37:04 -0700741 return (HasVendorPartition(target_files) and
Tao Bao481bab82017-12-21 11:23:09 -0800742 target_info.GetBuildProp("ro.treble.enabled") == "true")
Tao Baobcd1d162017-08-26 13:10:26 -0700743
744
Tao Bao481bab82017-12-21 11:23:09 -0800745def WriteFingerprintAssertion(script, target_info, source_info):
746 source_oem_props = source_info.oem_props
747 target_oem_props = target_info.oem_props
Michael Runge6e836112014-04-15 17:40:21 -0700748
Tao Bao481bab82017-12-21 11:23:09 -0800749 if source_oem_props is None and target_oem_props is None:
750 script.AssertSomeFingerprint(
751 source_info.fingerprint, target_info.fingerprint)
752 elif source_oem_props is not None and target_oem_props is not None:
753 script.AssertSomeThumbprint(
754 target_info.GetBuildProp("ro.build.thumbprint"),
755 source_info.GetBuildProp("ro.build.thumbprint"))
756 elif source_oem_props is None and target_oem_props is not None:
757 script.AssertFingerprintOrThumbprint(
758 source_info.fingerprint,
759 target_info.GetBuildProp("ro.build.thumbprint"))
760 else:
761 script.AssertFingerprintOrThumbprint(
762 target_info.fingerprint,
763 source_info.GetBuildProp("ro.build.thumbprint"))
Doug Zongker73ef8252009-07-23 15:12:53 -0700764
Doug Zongkerfc44a512014-08-26 13:10:25 -0700765
Yifan Hong9276cf02019-08-21 16:37:04 -0700766def CheckVintfIfTrebleEnabled(target_files, target_info):
767 """Checks compatibility info of the input target files.
Tao Bao21803d32017-04-19 10:16:09 -0700768
Yifan Hong9276cf02019-08-21 16:37:04 -0700769 Metadata used for compatibility verification is retrieved from target_zip.
Tao Bao21803d32017-04-19 10:16:09 -0700770
Yifan Hong9276cf02019-08-21 16:37:04 -0700771 Compatibility should only be checked for devices that have enabled
Tao Baobcd1d162017-08-26 13:10:26 -0700772 Treble support.
Tao Bao21803d32017-04-19 10:16:09 -0700773
774 Args:
Yifan Hong9276cf02019-08-21 16:37:04 -0700775 target_files: Path to zip file containing the source files to be included
776 for OTA. Can also be the path to extracted directory.
Tao Bao481bab82017-12-21 11:23:09 -0800777 target_info: The BuildInfo instance that holds the target build info.
Tao Bao21803d32017-04-19 10:16:09 -0700778 """
779
Tao Baobcd1d162017-08-26 13:10:26 -0700780 # Will only proceed if the target has enabled the Treble support (as well as
781 # having a /vendor partition).
Yifan Hong9276cf02019-08-21 16:37:04 -0700782 if not HasTrebleEnabled(target_files, target_info):
Tao Baobcd1d162017-08-26 13:10:26 -0700783 return
784
xunchangabfa2652019-02-19 16:27:10 -0800785 # Skip adding the compatibility package as a workaround for b/114240221. The
786 # compatibility will always fail on devices without qualified kernels.
787 if OPTIONS.skip_compatibility_check:
788 return
789
Yifan Hong9276cf02019-08-21 16:37:04 -0700790 if not check_target_files_vintf.CheckVintf(target_files, target_info):
791 raise RuntimeError("VINTF compatibility check failed")
Tao Bao21803d32017-04-19 10:16:09 -0700792
793
Tianjie Xuf67dd802019-05-20 17:50:36 -0700794def GetBlockDifferences(target_zip, source_zip, target_info, source_info,
795 device_specific):
796 """Returns a ordered dict of block differences with partition name as key."""
797
798 def GetIncrementalBlockDifferenceForPartition(name):
799 if not HasPartition(source_zip, name):
800 raise RuntimeError("can't generate incremental that adds {}".format(name))
801
802 partition_src = common.GetUserImage(name, OPTIONS.source_tmp, source_zip,
803 info_dict=source_info,
804 allow_shared_blocks=allow_shared_blocks)
805
806 hashtree_info_generator = verity_utils.CreateHashtreeInfoGenerator(
807 name, 4096, target_info)
808 partition_tgt = common.GetUserImage(name, OPTIONS.target_tmp, target_zip,
809 info_dict=target_info,
810 allow_shared_blocks=allow_shared_blocks,
811 hashtree_info_generator=
812 hashtree_info_generator)
813
814 # Check the first block of the source system partition for remount R/W only
815 # if the filesystem is ext4.
816 partition_source_info = source_info["fstab"]["/" + name]
817 check_first_block = partition_source_info.fs_type == "ext4"
818 # Disable using imgdiff for squashfs. 'imgdiff -z' expects input files to be
819 # in zip formats. However with squashfs, a) all files are compressed in LZ4;
820 # b) the blocks listed in block map may not contain all the bytes for a
821 # given file (because they're rounded to be 4K-aligned).
822 partition_target_info = target_info["fstab"]["/" + name]
823 disable_imgdiff = (partition_source_info.fs_type == "squashfs" or
824 partition_target_info.fs_type == "squashfs")
825 return common.BlockDifference(name, partition_src, partition_tgt,
826 check_first_block,
827 version=blockimgdiff_version,
828 disable_imgdiff=disable_imgdiff)
829
830 if source_zip:
831 # See notes in common.GetUserImage()
832 allow_shared_blocks = (source_info.get('ext4_share_dup_blocks') == "true" or
833 target_info.get('ext4_share_dup_blocks') == "true")
834 blockimgdiff_version = max(
835 int(i) for i in target_info.get(
836 "blockimgdiff_versions", "1").split(","))
837 assert blockimgdiff_version >= 3
838
839 block_diff_dict = collections.OrderedDict()
840 partition_names = ["system", "vendor", "product", "odm", "system_ext"]
841 for partition in partition_names:
842 if not HasPartition(target_zip, partition):
843 continue
844 # Full OTA update.
845 if not source_zip:
846 tgt = common.GetUserImage(partition, OPTIONS.input_tmp, target_zip,
847 info_dict=target_info,
848 reset_file_map=True)
849 block_diff_dict[partition] = common.BlockDifference(partition, tgt,
850 src=None)
851 # Incremental OTA update.
852 else:
853 block_diff_dict[partition] = GetIncrementalBlockDifferenceForPartition(
854 partition)
855 assert "system" in block_diff_dict
856
857 # Get the block diffs from the device specific script. If there is a
858 # duplicate block diff for a partition, ignore the diff in the generic script
859 # and use the one in the device specific script instead.
860 if source_zip:
861 device_specific_diffs = device_specific.IncrementalOTA_GetBlockDifferences()
862 function_name = "IncrementalOTA_GetBlockDifferences"
863 else:
864 device_specific_diffs = device_specific.FullOTA_GetBlockDifferences()
865 function_name = "FullOTA_GetBlockDifferences"
866
867 if device_specific_diffs:
868 assert all(isinstance(diff, common.BlockDifference)
869 for diff in device_specific_diffs), \
870 "{} is not returning a list of BlockDifference objects".format(
871 function_name)
872 for diff in device_specific_diffs:
873 if diff.partition in block_diff_dict:
874 logger.warning("Duplicate block difference found. Device specific block"
875 " diff for partition '%s' overrides the one in generic"
876 " script.", diff.partition)
877 block_diff_dict[diff.partition] = diff
878
879 return block_diff_dict
880
881
Tao Bao491d7e22018-02-21 13:17:22 -0800882def WriteFullOTAPackage(input_zip, output_file):
Tao Bao481bab82017-12-21 11:23:09 -0800883 target_info = BuildInfo(OPTIONS.info_dict, OPTIONS.oem_dicts)
Doug Zongkereef39442009-04-02 12:14:19 -0700884
Tao Bao481bab82017-12-21 11:23:09 -0800885 # We don't know what version it will be installed on top of. We expect the API
886 # just won't change very often. Similarly for fstab, it might have changed in
887 # the target build.
888 target_api_version = target_info["recovery_api_version"]
889 script = edify_generator.EdifyGenerator(target_api_version, target_info)
Michael Runge6e836112014-04-15 17:40:21 -0700890
Tao Bao481bab82017-12-21 11:23:09 -0800891 if target_info.oem_props and not OPTIONS.oem_no_mount:
892 target_info.WriteMountOemScript(script)
893
Tao Baodf3a48b2018-01-10 16:30:43 -0800894 metadata = GetPackageMetadata(target_info)
Doug Zongker2ea21062010-04-28 16:05:21 -0700895
Tao Bao491d7e22018-02-21 13:17:22 -0800896 if not OPTIONS.no_signing:
897 staging_file = common.MakeTempFile(suffix='.zip')
898 else:
899 staging_file = output_file
900
901 output_zip = zipfile.ZipFile(
902 staging_file, "w", compression=zipfile.ZIP_DEFLATED)
903
Doug Zongker05d3dea2009-06-22 11:32:31 -0700904 device_specific = common.DeviceSpecificParams(
905 input_zip=input_zip,
Tao Bao481bab82017-12-21 11:23:09 -0800906 input_version=target_api_version,
Doug Zongker05d3dea2009-06-22 11:32:31 -0700907 output_zip=output_zip,
908 script=script,
Doug Zongker2ea21062010-04-28 16:05:21 -0700909 input_tmp=OPTIONS.input_tmp,
Doug Zongker96a57e72010-09-26 14:57:41 -0700910 metadata=metadata,
911 info_dict=OPTIONS.info_dict)
Doug Zongker05d3dea2009-06-22 11:32:31 -0700912
Tao Bao457cbf62017-03-06 09:56:01 -0800913 assert HasRecoveryPatch(input_zip)
Doug Zongkerc9253822014-02-04 12:17:58 -0800914
Tao Bao481bab82017-12-21 11:23:09 -0800915 # Assertions (e.g. downgrade check, device properties check).
916 ts = target_info.GetBuildProp("ro.build.date.utc")
917 ts_text = target_info.GetBuildProp("ro.build.date")
Elliott Hughesd8a52f92016-06-20 14:35:47 -0700918 script.AssertOlderBuild(ts, ts_text)
Doug Zongkereef39442009-04-02 12:14:19 -0700919
Tao Bao481bab82017-12-21 11:23:09 -0800920 target_info.WriteDeviceAssertions(script, OPTIONS.oem_no_mount)
Doug Zongker05d3dea2009-06-22 11:32:31 -0700921 device_specific.FullOTA_Assertions()
Doug Zongker9b23f2c2013-11-25 14:44:12 -0800922
Tianjie Xuf67dd802019-05-20 17:50:36 -0700923 block_diff_dict = GetBlockDifferences(target_zip=input_zip, source_zip=None,
924 target_info=target_info,
925 source_info=None,
926 device_specific=device_specific)
927
Doug Zongker9b23f2c2013-11-25 14:44:12 -0800928 # Two-step package strategy (in chronological order, which is *not*
929 # the order in which the generated script has things):
930 #
931 # if stage is not "2/3" or "3/3":
932 # write recovery image to boot partition
933 # set stage to "2/3"
934 # reboot to boot partition and restart recovery
935 # else if stage is "2/3":
936 # write recovery image to recovery partition
937 # set stage to "3/3"
938 # reboot to recovery partition and restart recovery
939 # else:
940 # (stage must be "3/3")
941 # set stage to ""
942 # do normal full package installation:
943 # wipe and install system, boot image, etc.
944 # set up system to update recovery partition on first boot
Dan Albert8b72aef2015-03-23 19:13:21 -0700945 # complete script normally
946 # (allow recovery to mark itself finished and reboot)
Doug Zongker9b23f2c2013-11-25 14:44:12 -0800947
948 recovery_img = common.GetBootableImage("recovery.img", "recovery.img",
949 OPTIONS.input_tmp, "RECOVERY")
950 if OPTIONS.two_step:
Tao Bao481bab82017-12-21 11:23:09 -0800951 if not target_info.get("multistage_support"):
Doug Zongker9b23f2c2013-11-25 14:44:12 -0800952 assert False, "two-step packages not supported by this build"
Tao Bao481bab82017-12-21 11:23:09 -0800953 fs = target_info["fstab"]["/misc"]
Doug Zongker9b23f2c2013-11-25 14:44:12 -0800954 assert fs.fs_type.upper() == "EMMC", \
955 "two-step packages only supported on devices with EMMC /misc partitions"
956 bcb_dev = {"bcb_dev": fs.device}
957 common.ZipWriteStr(output_zip, "recovery.img", recovery_img.data)
958 script.AppendExtra("""
Michael Rungefb8886d2014-10-23 13:51:04 -0700959if get_stage("%(bcb_dev)s") == "2/3" then
Doug Zongker9b23f2c2013-11-25 14:44:12 -0800960""" % bcb_dev)
Tao Baod42e97e2016-11-30 12:11:57 -0800961
962 # Stage 2/3: Write recovery image to /recovery (currently running /boot).
963 script.Comment("Stage 2/3")
Doug Zongker9b23f2c2013-11-25 14:44:12 -0800964 script.WriteRawImage("/recovery", "recovery.img")
965 script.AppendExtra("""
966set_stage("%(bcb_dev)s", "3/3");
967reboot_now("%(bcb_dev)s", "recovery");
Michael Rungefb8886d2014-10-23 13:51:04 -0700968else if get_stage("%(bcb_dev)s") == "3/3" then
Doug Zongker9b23f2c2013-11-25 14:44:12 -0800969""" % bcb_dev)
970
Tao Baod42e97e2016-11-30 12:11:57 -0800971 # Stage 3/3: Make changes.
972 script.Comment("Stage 3/3")
973
Tao Bao6c55a8a2015-04-08 15:30:27 -0700974 # Dump fingerprints
Tao Bao481bab82017-12-21 11:23:09 -0800975 script.Print("Target: {}".format(target_info.fingerprint))
Tao Bao6c55a8a2015-04-08 15:30:27 -0700976
Doug Zongkere5ff5902012-01-17 10:55:37 -0800977 device_specific.FullOTA_InstallBegin()
Doug Zongker171f1cd2009-06-15 22:36:37 -0700978
Tianjie Xuf67dd802019-05-20 17:50:36 -0700979 # All other partitions as well as the data wipe use 10% of the progress, and
980 # the update of the system partition takes the remaining progress.
981 system_progress = 0.9 - (len(block_diff_dict) - 1) * 0.1
Doug Zongkerdbfaae52009-04-21 17:12:54 -0700982 if OPTIONS.wipe_user_data:
Doug Zongker01ce19c2014-02-04 13:48:15 -0800983 system_progress -= 0.1
Tianjie Xuf67dd802019-05-20 17:50:36 -0700984 progress_dict = {partition: 0.1 for partition in block_diff_dict}
985 progress_dict["system"] = system_progress
Doug Zongkerc8b4e842014-06-16 15:16:31 -0700986
Yifan Hong10c530d2018-12-27 17:34:18 -0800987 if target_info.get('use_dynamic_partitions') == "true":
988 # Use empty source_info_dict to indicate that all partitions / groups must
989 # be re-added.
990 dynamic_partitions_diff = common.DynamicPartitionsDifference(
991 info_dict=OPTIONS.info_dict,
Tianjie Xuf67dd802019-05-20 17:50:36 -0700992 block_diffs=block_diff_dict.values(),
Yifan Hong10c530d2018-12-27 17:34:18 -0800993 progress_dict=progress_dict)
994 dynamic_partitions_diff.WriteScript(script, output_zip,
995 write_verify_script=OPTIONS.verify)
996 else:
Tianjie Xuf67dd802019-05-20 17:50:36 -0700997 for block_diff in block_diff_dict.values():
Yifan Hong10c530d2018-12-27 17:34:18 -0800998 block_diff.WriteScript(script, output_zip,
999 progress=progress_dict.get(block_diff.partition),
1000 write_verify_script=OPTIONS.verify)
Doug Zongker73ef8252009-07-23 15:12:53 -07001001
Yifan Hong9276cf02019-08-21 16:37:04 -07001002 CheckVintfIfTrebleEnabled(OPTIONS.input_tmp, target_info)
Tao Baobcd1d162017-08-26 13:10:26 -07001003
Yifan Hong10c530d2018-12-27 17:34:18 -08001004 boot_img = common.GetBootableImage(
1005 "boot.img", "boot.img", OPTIONS.input_tmp, "BOOT")
Tao Bao481bab82017-12-21 11:23:09 -08001006 common.CheckSize(boot_img.data, "boot.img", target_info)
Doug Zongker73ef8252009-07-23 15:12:53 -07001007 common.ZipWriteStr(output_zip, "boot.img", boot_img.data)
Doug Zongkerc494d7c2009-06-18 08:43:44 -07001008
Doug Zongker9ce0fb62010-09-20 18:04:41 -07001009 script.WriteRawImage("/boot", "boot.img")
Doug Zongker05d3dea2009-06-22 11:32:31 -07001010
Tianjie Xuf67dd802019-05-20 17:50:36 -07001011 script.ShowProgress(0.1, 10)
Doug Zongker05d3dea2009-06-22 11:32:31 -07001012 device_specific.FullOTA_InstallEnd()
Doug Zongkereef39442009-04-02 12:14:19 -07001013
Doug Zongker1c390a22009-05-14 19:06:36 -07001014 if OPTIONS.extra_script is not None:
Doug Zongkerc494d7c2009-06-18 08:43:44 -07001015 script.AppendExtra(OPTIONS.extra_script)
Doug Zongker1c390a22009-05-14 19:06:36 -07001016
Doug Zongker14833602010-02-02 13:12:04 -08001017 script.UnmountAll()
Doug Zongker9b23f2c2013-11-25 14:44:12 -08001018
Doug Zongker922206e2014-03-04 13:16:24 -08001019 if OPTIONS.wipe_user_data:
1020 script.ShowProgress(0.1, 10)
1021 script.FormatPartition("/data")
Doug Zongkerc8b4e842014-06-16 15:16:31 -07001022
Doug Zongker9b23f2c2013-11-25 14:44:12 -08001023 if OPTIONS.two_step:
1024 script.AppendExtra("""
1025set_stage("%(bcb_dev)s", "");
1026""" % bcb_dev)
1027 script.AppendExtra("else\n")
Tao Baod42e97e2016-11-30 12:11:57 -08001028
1029 # Stage 1/3: Nothing to verify for full OTA. Write recovery image to /boot.
1030 script.Comment("Stage 1/3")
1031 _WriteRecoveryImageToBoot(script, output_zip)
1032
Doug Zongker9b23f2c2013-11-25 14:44:12 -08001033 script.AppendExtra("""
1034set_stage("%(bcb_dev)s", "2/3");
1035reboot_now("%(bcb_dev)s", "");
1036endif;
1037endif;
1038""" % bcb_dev)
Tao Baod8d14be2016-02-04 14:26:02 -08001039
Tao Bao5d182562016-02-23 11:38:39 -08001040 script.SetProgress(1)
1041 script.AddToZip(input_zip, output_zip, input_path=OPTIONS.updater_binary)
Tao Baod8d14be2016-02-04 14:26:02 -08001042 metadata["ota-required-cache"] = str(script.required_cache)
Tao Bao491d7e22018-02-21 13:17:22 -08001043
1044 # We haven't written the metadata entry, which will be done in
1045 # FinalizeMetadata.
1046 common.ZipClose(output_zip)
1047
1048 needed_property_files = (
1049 NonAbOtaPropertyFiles(),
1050 )
1051 FinalizeMetadata(metadata, staging_file, output_file, needed_property_files)
Doug Zongker2ea21062010-04-28 16:05:21 -07001052
Doug Zongkerfc44a512014-08-26 13:10:25 -07001053
xunchang1cfe2512019-02-19 14:14:48 -08001054def WriteMetadata(metadata, output):
1055 """Writes the metadata to the zip archive or a file.
1056
1057 Args:
1058 metadata: The metadata dict for the package.
1059 output: A ZipFile object or a string of the output file path.
1060 """
1061
Tao Bao59cf0c52019-06-25 10:04:24 -07001062 value = "".join(["%s=%s\n" % kv for kv in sorted(metadata.items())])
xunchang1cfe2512019-02-19 14:14:48 -08001063 if isinstance(output, zipfile.ZipFile):
1064 common.ZipWriteStr(output, METADATA_NAME, value,
1065 compress_type=zipfile.ZIP_STORED)
1066 return
1067
1068 with open(output, 'w') as f:
1069 f.write(value)
Doug Zongkereef39442009-04-02 12:14:19 -07001070
Doug Zongkerfc44a512014-08-26 13:10:25 -07001071
Tao Bao481bab82017-12-21 11:23:09 -08001072def HandleDowngradeMetadata(metadata, target_info, source_info):
Tao Baob31892e2017-02-07 11:21:17 -08001073 # Only incremental OTAs are allowed to reach here.
1074 assert OPTIONS.incremental_source is not None
1075
Tao Bao481bab82017-12-21 11:23:09 -08001076 post_timestamp = target_info.GetBuildProp("ro.build.date.utc")
1077 pre_timestamp = source_info.GetBuildProp("ro.build.date.utc")
Tao Bao59cf0c52019-06-25 10:04:24 -07001078 is_downgrade = int(post_timestamp) < int(pre_timestamp)
Tao Baob31892e2017-02-07 11:21:17 -08001079
1080 if OPTIONS.downgrade:
Tao Baob31892e2017-02-07 11:21:17 -08001081 if not is_downgrade:
Tao Baofaa8e0b2018-04-12 14:31:43 -07001082 raise RuntimeError(
1083 "--downgrade or --override_timestamp specified but no downgrade "
1084 "detected: pre: %s, post: %s" % (pre_timestamp, post_timestamp))
Tao Bao3e6161a2017-02-28 11:48:48 -08001085 metadata["ota-downgrade"] = "yes"
Tao Baob31892e2017-02-07 11:21:17 -08001086 else:
1087 if is_downgrade:
Tao Baofaa8e0b2018-04-12 14:31:43 -07001088 raise RuntimeError(
1089 "Downgrade detected based on timestamp check: pre: %s, post: %s. "
1090 "Need to specify --override_timestamp OR --downgrade to allow "
1091 "building the incremental." % (pre_timestamp, post_timestamp))
Tao Baob31892e2017-02-07 11:21:17 -08001092
1093
Tao Baodf3a48b2018-01-10 16:30:43 -08001094def GetPackageMetadata(target_info, source_info=None):
1095 """Generates and returns the metadata dict.
1096
1097 It generates a dict() that contains the info to be written into an OTA
1098 package (META-INF/com/android/metadata). It also handles the detection of
Tao Baofaa8e0b2018-04-12 14:31:43 -07001099 downgrade / data wipe based on the global options.
Tao Baodf3a48b2018-01-10 16:30:43 -08001100
1101 Args:
1102 target_info: The BuildInfo instance that holds the target build info.
1103 source_info: The BuildInfo instance that holds the source build info, or
1104 None if generating full OTA.
1105
1106 Returns:
1107 A dict to be written into package metadata entry.
1108 """
1109 assert isinstance(target_info, BuildInfo)
1110 assert source_info is None or isinstance(source_info, BuildInfo)
1111
1112 metadata = {
1113 'post-build' : target_info.fingerprint,
1114 'post-build-incremental' : target_info.GetBuildProp(
1115 'ro.build.version.incremental'),
Tao Bao35dc2552018-02-01 13:18:00 -08001116 'post-sdk-level' : target_info.GetBuildProp(
1117 'ro.build.version.sdk'),
1118 'post-security-patch-level' : target_info.GetBuildProp(
1119 'ro.build.version.security_patch'),
Tao Baodf3a48b2018-01-10 16:30:43 -08001120 }
1121
1122 if target_info.is_ab:
1123 metadata['ota-type'] = 'AB'
1124 metadata['ota-required-cache'] = '0'
1125 else:
1126 metadata['ota-type'] = 'BLOCK'
1127
1128 if OPTIONS.wipe_user_data:
1129 metadata['ota-wipe'] = 'yes'
1130
Tao Bao393eeb42019-03-06 16:00:38 -08001131 if OPTIONS.retrofit_dynamic_partitions:
1132 metadata['ota-retrofit-dynamic-partitions'] = 'yes'
1133
Tao Baodf3a48b2018-01-10 16:30:43 -08001134 is_incremental = source_info is not None
1135 if is_incremental:
1136 metadata['pre-build'] = source_info.fingerprint
1137 metadata['pre-build-incremental'] = source_info.GetBuildProp(
1138 'ro.build.version.incremental')
1139 metadata['pre-device'] = source_info.device
1140 else:
1141 metadata['pre-device'] = target_info.device
1142
Tao Baofaa8e0b2018-04-12 14:31:43 -07001143 # Use the actual post-timestamp, even for a downgrade case.
1144 metadata['post-timestamp'] = target_info.GetBuildProp('ro.build.date.utc')
1145
1146 # Detect downgrades and set up downgrade flags accordingly.
Tao Baodf3a48b2018-01-10 16:30:43 -08001147 if is_incremental:
1148 HandleDowngradeMetadata(metadata, target_info, source_info)
Tao Baodf3a48b2018-01-10 16:30:43 -08001149
1150 return metadata
1151
1152
Tao Baod3fc38a2018-03-08 16:09:01 -08001153class PropertyFiles(object):
1154 """A class that computes the property-files string for an OTA package.
1155
1156 A property-files string is a comma-separated string that contains the
1157 offset/size info for an OTA package. The entries, which must be ZIP_STORED,
1158 can be fetched directly with the package URL along with the offset/size info.
1159 These strings can be used for streaming A/B OTAs, or allowing an updater to
1160 download package metadata entry directly, without paying the cost of
1161 downloading entire package.
Tao Baofe5b69a2018-03-02 09:47:43 -08001162
Tao Baocc8e2662018-03-01 19:30:00 -08001163 Computing the final property-files string requires two passes. Because doing
1164 the whole package signing (with signapk.jar) will possibly reorder the ZIP
1165 entries, which may in turn invalidate earlier computed ZIP entry offset/size
1166 values.
1167
1168 This class provides functions to be called for each pass. The general flow is
1169 as follows.
1170
Tao Baod3fc38a2018-03-08 16:09:01 -08001171 property_files = PropertyFiles()
Tao Baocc8e2662018-03-01 19:30:00 -08001172 # The first pass, which writes placeholders before doing initial signing.
1173 property_files.Compute()
1174 SignOutput()
1175
1176 # The second pass, by replacing the placeholders with actual data.
1177 property_files.Finalize()
1178 SignOutput()
1179
1180 And the caller can additionally verify the final result.
1181
1182 property_files.Verify()
Tao Baofe5b69a2018-03-02 09:47:43 -08001183 """
1184
Tao Baocc8e2662018-03-01 19:30:00 -08001185 def __init__(self):
Tao Baod3fc38a2018-03-08 16:09:01 -08001186 self.name = None
1187 self.required = ()
1188 self.optional = ()
Tao Baofe5b69a2018-03-02 09:47:43 -08001189
Tao Baocc8e2662018-03-01 19:30:00 -08001190 def Compute(self, input_zip):
1191 """Computes and returns a property-files string with placeholders.
Tao Baofe5b69a2018-03-02 09:47:43 -08001192
Tao Baocc8e2662018-03-01 19:30:00 -08001193 We reserve extra space for the offset and size of the metadata entry itself,
1194 although we don't know the final values until the package gets signed.
Tao Baofe5b69a2018-03-02 09:47:43 -08001195
Tao Baocc8e2662018-03-01 19:30:00 -08001196 Args:
1197 input_zip: The input ZIP file.
Tao Baofe5b69a2018-03-02 09:47:43 -08001198
Tao Baocc8e2662018-03-01 19:30:00 -08001199 Returns:
1200 A string with placeholders for the metadata offset/size info, e.g.
1201 "payload.bin:679:343,payload_properties.txt:378:45,metadata: ".
1202 """
Zhomart Mukhamejanov603655f2018-05-04 12:35:09 -07001203 return self.GetPropertyFilesString(input_zip, reserve_space=True)
Tao Baofe5b69a2018-03-02 09:47:43 -08001204
Tao Baod2ce2ed2018-03-16 12:59:42 -07001205 class InsufficientSpaceException(Exception):
1206 pass
1207
Tao Baocc8e2662018-03-01 19:30:00 -08001208 def Finalize(self, input_zip, reserved_length):
1209 """Finalizes a property-files string with actual METADATA offset/size info.
1210
1211 The input ZIP file has been signed, with the ZIP entries in the desired
1212 place (signapk.jar will possibly reorder the ZIP entries). Now we compute
1213 the ZIP entry offsets and construct the property-files string with actual
1214 data. Note that during this process, we must pad the property-files string
1215 to the reserved length, so that the METADATA entry size remains the same.
1216 Otherwise the entries' offsets and sizes may change again.
1217
1218 Args:
1219 input_zip: The input ZIP file.
1220 reserved_length: The reserved length of the property-files string during
1221 the call to Compute(). The final string must be no more than this
1222 size.
1223
1224 Returns:
1225 A property-files string including the metadata offset/size info, e.g.
1226 "payload.bin:679:343,payload_properties.txt:378:45,metadata:69:379 ".
1227
1228 Raises:
Tao Baod2ce2ed2018-03-16 12:59:42 -07001229 InsufficientSpaceException: If the reserved length is insufficient to hold
1230 the final string.
Tao Baocc8e2662018-03-01 19:30:00 -08001231 """
Zhomart Mukhamejanov603655f2018-05-04 12:35:09 -07001232 result = self.GetPropertyFilesString(input_zip, reserve_space=False)
Tao Baod2ce2ed2018-03-16 12:59:42 -07001233 if len(result) > reserved_length:
1234 raise self.InsufficientSpaceException(
1235 'Insufficient reserved space: reserved={}, actual={}'.format(
1236 reserved_length, len(result)))
1237
Tao Baocc8e2662018-03-01 19:30:00 -08001238 result += ' ' * (reserved_length - len(result))
1239 return result
1240
1241 def Verify(self, input_zip, expected):
1242 """Verifies the input ZIP file contains the expected property-files string.
1243
1244 Args:
1245 input_zip: The input ZIP file.
1246 expected: The property-files string that's computed from Finalize().
1247
1248 Raises:
1249 AssertionError: On finding a mismatch.
1250 """
Zhomart Mukhamejanov603655f2018-05-04 12:35:09 -07001251 actual = self.GetPropertyFilesString(input_zip)
Tao Baocc8e2662018-03-01 19:30:00 -08001252 assert actual == expected, \
1253 "Mismatching streaming metadata: {} vs {}.".format(actual, expected)
1254
Zhomart Mukhamejanov603655f2018-05-04 12:35:09 -07001255 def GetPropertyFilesString(self, zip_file, reserve_space=False):
1256 """
1257 Constructs the property-files string per request.
1258
1259 Args:
1260 zip_file: The input ZIP file.
1261 reserved_length: The reserved length of the property-files string.
1262
1263 Returns:
1264 A property-files string including the metadata offset/size info, e.g.
1265 "payload.bin:679:343,payload_properties.txt:378:45,metadata: ".
1266 """
Tao Baocc8e2662018-03-01 19:30:00 -08001267
1268 def ComputeEntryOffsetSize(name):
1269 """Computes the zip entry offset and size."""
1270 info = zip_file.getinfo(name)
Shashikant Baviskar338856f2018-04-12 12:11:22 +09001271 offset = info.header_offset
1272 offset += zipfile.sizeFileHeader
1273 offset += len(info.extra) + len(info.filename)
Tao Baocc8e2662018-03-01 19:30:00 -08001274 size = info.file_size
1275 return '%s:%d:%d' % (os.path.basename(name), offset, size)
1276
1277 tokens = []
Tao Bao85f16982018-03-08 16:28:33 -08001278 tokens.extend(self._GetPrecomputed(zip_file))
Tao Baocc8e2662018-03-01 19:30:00 -08001279 for entry in self.required:
1280 tokens.append(ComputeEntryOffsetSize(entry))
1281 for entry in self.optional:
1282 if entry in zip_file.namelist():
1283 tokens.append(ComputeEntryOffsetSize(entry))
1284
1285 # 'META-INF/com/android/metadata' is required. We don't know its actual
1286 # offset and length (as well as the values for other entries). So we reserve
Tao Baod2ce2ed2018-03-16 12:59:42 -07001287 # 15-byte as a placeholder ('offset:length'), which is sufficient to cover
1288 # the space for metadata entry. Because 'offset' allows a max of 10-digit
1289 # (i.e. ~9 GiB), with a max of 4-digit for the length. Note that all the
1290 # reserved space serves the metadata entry only.
Tao Baocc8e2662018-03-01 19:30:00 -08001291 if reserve_space:
Tao Baod2ce2ed2018-03-16 12:59:42 -07001292 tokens.append('metadata:' + ' ' * 15)
Tao Baocc8e2662018-03-01 19:30:00 -08001293 else:
1294 tokens.append(ComputeEntryOffsetSize(METADATA_NAME))
1295
1296 return ','.join(tokens)
Tao Baofe5b69a2018-03-02 09:47:43 -08001297
Tao Bao85f16982018-03-08 16:28:33 -08001298 def _GetPrecomputed(self, input_zip):
1299 """Computes the additional tokens to be included into the property-files.
1300
1301 This applies to tokens without actual ZIP entries, such as
1302 payload_metadadata.bin. We want to expose the offset/size to updaters, so
1303 that they can download the payload metadata directly with the info.
1304
1305 Args:
1306 input_zip: The input zip file.
1307
1308 Returns:
1309 A list of strings (tokens) to be added to the property-files string.
1310 """
1311 # pylint: disable=no-self-use
1312 # pylint: disable=unused-argument
1313 return []
1314
Tao Baofe5b69a2018-03-02 09:47:43 -08001315
Tao Baod3fc38a2018-03-08 16:09:01 -08001316class StreamingPropertyFiles(PropertyFiles):
1317 """A subclass for computing the property-files for streaming A/B OTAs."""
1318
1319 def __init__(self):
1320 super(StreamingPropertyFiles, self).__init__()
1321 self.name = 'ota-streaming-property-files'
1322 self.required = (
1323 # payload.bin and payload_properties.txt must exist.
1324 'payload.bin',
1325 'payload_properties.txt',
1326 )
1327 self.optional = (
Tianjie Xu4c05f4a2018-09-14 16:24:41 -07001328 # care_map is available only if dm-verity is enabled.
1329 'care_map.pb',
Tao Baod3fc38a2018-03-08 16:09:01 -08001330 'care_map.txt',
1331 # compatibility.zip is available only if target supports Treble.
1332 'compatibility.zip',
1333 )
1334
1335
Tao Bao85f16982018-03-08 16:28:33 -08001336class AbOtaPropertyFiles(StreamingPropertyFiles):
1337 """The property-files for A/B OTA that includes payload_metadata.bin info.
1338
1339 Since P, we expose one more token (aka property-file), in addition to the ones
1340 for streaming A/B OTA, for a virtual entry of 'payload_metadata.bin'.
1341 'payload_metadata.bin' is the header part of a payload ('payload.bin'), which
1342 doesn't exist as a separate ZIP entry, but can be used to verify if the
1343 payload can be applied on the given device.
1344
1345 For backward compatibility, we keep both of the 'ota-streaming-property-files'
1346 and the newly added 'ota-property-files' in P. The new token will only be
1347 available in 'ota-property-files'.
1348 """
1349
1350 def __init__(self):
1351 super(AbOtaPropertyFiles, self).__init__()
1352 self.name = 'ota-property-files'
1353
1354 def _GetPrecomputed(self, input_zip):
1355 offset, size = self._GetPayloadMetadataOffsetAndSize(input_zip)
1356 return ['payload_metadata.bin:{}:{}'.format(offset, size)]
1357
1358 @staticmethod
1359 def _GetPayloadMetadataOffsetAndSize(input_zip):
1360 """Computes the offset and size of the payload metadata for a given package.
1361
1362 (From system/update_engine/update_metadata.proto)
1363 A delta update file contains all the deltas needed to update a system from
1364 one specific version to another specific version. The update format is
1365 represented by this struct pseudocode:
1366
1367 struct delta_update_file {
1368 char magic[4] = "CrAU";
1369 uint64 file_format_version;
1370 uint64 manifest_size; // Size of protobuf DeltaArchiveManifest
1371
1372 // Only present if format_version > 1:
1373 uint32 metadata_signature_size;
1374
1375 // The Bzip2 compressed DeltaArchiveManifest
1376 char manifest[metadata_signature_size];
1377
1378 // The signature of the metadata (from the beginning of the payload up to
1379 // this location, not including the signature itself). This is a
1380 // serialized Signatures message.
1381 char medatada_signature_message[metadata_signature_size];
1382
1383 // Data blobs for files, no specific format. The specific offset
1384 // and length of each data blob is recorded in the DeltaArchiveManifest.
1385 struct {
1386 char data[];
1387 } blobs[];
1388
1389 // These two are not signed:
1390 uint64 payload_signatures_message_size;
1391 char payload_signatures_message[];
1392 };
1393
1394 'payload-metadata.bin' contains all the bytes from the beginning of the
1395 payload, till the end of 'medatada_signature_message'.
1396 """
1397 payload_info = input_zip.getinfo('payload.bin')
Shashikant Baviskar338856f2018-04-12 12:11:22 +09001398 payload_offset = payload_info.header_offset
1399 payload_offset += zipfile.sizeFileHeader
1400 payload_offset += len(payload_info.extra) + len(payload_info.filename)
Tao Bao85f16982018-03-08 16:28:33 -08001401 payload_size = payload_info.file_size
1402
Tao Bao59cf0c52019-06-25 10:04:24 -07001403 with input_zip.open('payload.bin') as payload_fp:
Tao Bao85f16982018-03-08 16:28:33 -08001404 header_bin = payload_fp.read(24)
1405
1406 # network byte order (big-endian)
1407 header = struct.unpack("!IQQL", header_bin)
1408
1409 # 'CrAU'
1410 magic = header[0]
1411 assert magic == 0x43724155, "Invalid magic: {:x}".format(magic)
1412
1413 manifest_size = header[2]
1414 metadata_signature_size = header[3]
1415 metadata_total = 24 + manifest_size + metadata_signature_size
1416 assert metadata_total < payload_size
1417
1418 return (payload_offset, metadata_total)
1419
1420
Tao Bao491d7e22018-02-21 13:17:22 -08001421class NonAbOtaPropertyFiles(PropertyFiles):
1422 """The property-files for non-A/B OTA.
1423
1424 For non-A/B OTA, the property-files string contains the info for METADATA
1425 entry, with which a system updater can be fetched the package metadata prior
1426 to downloading the entire package.
1427 """
1428
1429 def __init__(self):
1430 super(NonAbOtaPropertyFiles, self).__init__()
1431 self.name = 'ota-property-files'
1432
1433
Tao Baod3fc38a2018-03-08 16:09:01 -08001434def FinalizeMetadata(metadata, input_file, output_file, needed_property_files):
Tao Baofe5b69a2018-03-02 09:47:43 -08001435 """Finalizes the metadata and signs an A/B OTA package.
1436
1437 In order to stream an A/B OTA package, we need 'ota-streaming-property-files'
1438 that contains the offsets and sizes for the ZIP entries. An example
1439 property-files string is as follows.
1440
1441 "payload.bin:679:343,payload_properties.txt:378:45,metadata:69:379"
1442
1443 OTA server can pass down this string, in addition to the package URL, to the
1444 system update client. System update client can then fetch individual ZIP
1445 entries (ZIP_STORED) directly at the given offset of the URL.
1446
1447 Args:
1448 metadata: The metadata dict for the package.
1449 input_file: The input ZIP filename that doesn't contain the package METADATA
1450 entry yet.
1451 output_file: The final output ZIP filename.
Tao Baod3fc38a2018-03-08 16:09:01 -08001452 needed_property_files: The list of PropertyFiles' to be generated.
Tao Baofe5b69a2018-03-02 09:47:43 -08001453 """
Tao Baofe5b69a2018-03-02 09:47:43 -08001454
Tao Baod2ce2ed2018-03-16 12:59:42 -07001455 def ComputeAllPropertyFiles(input_file, needed_property_files):
1456 # Write the current metadata entry with placeholders.
1457 with zipfile.ZipFile(input_file) as input_zip:
1458 for property_files in needed_property_files:
1459 metadata[property_files.name] = property_files.Compute(input_zip)
1460 namelist = input_zip.namelist()
Tao Baofe5b69a2018-03-02 09:47:43 -08001461
Tao Baod2ce2ed2018-03-16 12:59:42 -07001462 if METADATA_NAME in namelist:
1463 common.ZipDelete(input_file, METADATA_NAME)
1464 output_zip = zipfile.ZipFile(input_file, 'a')
1465 WriteMetadata(metadata, output_zip)
1466 common.ZipClose(output_zip)
1467
1468 if OPTIONS.no_signing:
1469 return input_file
1470
Tao Bao491d7e22018-02-21 13:17:22 -08001471 prelim_signing = common.MakeTempFile(suffix='.zip')
1472 SignOutput(input_file, prelim_signing)
Tao Baod2ce2ed2018-03-16 12:59:42 -07001473 return prelim_signing
Tao Baofe5b69a2018-03-02 09:47:43 -08001474
Tao Baod2ce2ed2018-03-16 12:59:42 -07001475 def FinalizeAllPropertyFiles(prelim_signing, needed_property_files):
1476 with zipfile.ZipFile(prelim_signing) as prelim_signing_zip:
1477 for property_files in needed_property_files:
1478 metadata[property_files.name] = property_files.Finalize(
1479 prelim_signing_zip, len(metadata[property_files.name]))
1480
1481 # SignOutput(), which in turn calls signapk.jar, will possibly reorder the ZIP
1482 # entries, as well as padding the entry headers. We do a preliminary signing
1483 # (with an incomplete metadata entry) to allow that to happen. Then compute
1484 # the ZIP entry offsets, write back the final metadata and do the final
1485 # signing.
1486 prelim_signing = ComputeAllPropertyFiles(input_file, needed_property_files)
1487 try:
1488 FinalizeAllPropertyFiles(prelim_signing, needed_property_files)
1489 except PropertyFiles.InsufficientSpaceException:
1490 # Even with the preliminary signing, the entry orders may change
1491 # dramatically, which leads to insufficiently reserved space during the
1492 # first call to ComputeAllPropertyFiles(). In that case, we redo all the
1493 # preliminary signing works, based on the already ordered ZIP entries, to
1494 # address the issue.
1495 prelim_signing = ComputeAllPropertyFiles(
1496 prelim_signing, needed_property_files)
1497 FinalizeAllPropertyFiles(prelim_signing, needed_property_files)
Tao Baofe5b69a2018-03-02 09:47:43 -08001498
1499 # Replace the METADATA entry.
1500 common.ZipDelete(prelim_signing, METADATA_NAME)
Tao Baod2ce2ed2018-03-16 12:59:42 -07001501 output_zip = zipfile.ZipFile(prelim_signing, 'a')
Tao Baofe5b69a2018-03-02 09:47:43 -08001502 WriteMetadata(metadata, output_zip)
1503 common.ZipClose(output_zip)
1504
1505 # Re-sign the package after updating the metadata entry.
Tao Bao491d7e22018-02-21 13:17:22 -08001506 if OPTIONS.no_signing:
1507 output_file = prelim_signing
1508 else:
1509 SignOutput(prelim_signing, output_file)
Tao Baofe5b69a2018-03-02 09:47:43 -08001510
1511 # Reopen the final signed zip to double check the streaming metadata.
Tao Baod2ce2ed2018-03-16 12:59:42 -07001512 with zipfile.ZipFile(output_file) as output_zip:
Tao Baod3fc38a2018-03-08 16:09:01 -08001513 for property_files in needed_property_files:
1514 property_files.Verify(output_zip, metadata[property_files.name].strip())
Tao Baofe5b69a2018-03-02 09:47:43 -08001515
xunchang1cfe2512019-02-19 14:14:48 -08001516 # If requested, dump the metadata to a separate file.
1517 output_metadata_path = OPTIONS.output_metadata_path
1518 if output_metadata_path:
1519 WriteMetadata(metadata, output_metadata_path)
1520
Tao Baofe5b69a2018-03-02 09:47:43 -08001521
Tao Bao491d7e22018-02-21 13:17:22 -08001522def WriteBlockIncrementalOTAPackage(target_zip, source_zip, output_file):
Tao Bao481bab82017-12-21 11:23:09 -08001523 target_info = BuildInfo(OPTIONS.target_info_dict, OPTIONS.oem_dicts)
1524 source_info = BuildInfo(OPTIONS.source_info_dict, OPTIONS.oem_dicts)
Geremy Condra36bd3652014-02-06 19:45:10 -08001525
Tao Bao481bab82017-12-21 11:23:09 -08001526 target_api_version = target_info["recovery_api_version"]
1527 source_api_version = source_info["recovery_api_version"]
1528 if source_api_version == 0:
Tao Bao32fcdab2018-10-12 10:30:39 -07001529 logger.warning(
1530 "Generating edify script for a source that can't install it.")
Geremy Condra36bd3652014-02-06 19:45:10 -08001531
Tao Bao481bab82017-12-21 11:23:09 -08001532 script = edify_generator.EdifyGenerator(
1533 source_api_version, target_info, fstab=source_info["fstab"])
1534
1535 if target_info.oem_props or source_info.oem_props:
1536 if not OPTIONS.oem_no_mount:
1537 source_info.WriteMountOemScript(script)
Tao Bao3806c232015-07-05 21:08:33 -07001538
Tao Baodf3a48b2018-01-10 16:30:43 -08001539 metadata = GetPackageMetadata(target_info, source_info)
Tao Bao5d182562016-02-23 11:38:39 -08001540
Tao Bao491d7e22018-02-21 13:17:22 -08001541 if not OPTIONS.no_signing:
1542 staging_file = common.MakeTempFile(suffix='.zip')
1543 else:
1544 staging_file = output_file
1545
1546 output_zip = zipfile.ZipFile(
1547 staging_file, "w", compression=zipfile.ZIP_DEFLATED)
1548
Geremy Condra36bd3652014-02-06 19:45:10 -08001549 device_specific = common.DeviceSpecificParams(
1550 source_zip=source_zip,
Tao Bao481bab82017-12-21 11:23:09 -08001551 source_version=source_api_version,
Yifan Hong8a66a712019-04-04 15:37:57 -07001552 source_tmp=OPTIONS.source_tmp,
Geremy Condra36bd3652014-02-06 19:45:10 -08001553 target_zip=target_zip,
Tao Bao481bab82017-12-21 11:23:09 -08001554 target_version=target_api_version,
Yifan Hong8a66a712019-04-04 15:37:57 -07001555 target_tmp=OPTIONS.target_tmp,
Geremy Condra36bd3652014-02-06 19:45:10 -08001556 output_zip=output_zip,
1557 script=script,
1558 metadata=metadata,
Tao Bao481bab82017-12-21 11:23:09 -08001559 info_dict=source_info)
Geremy Condra36bd3652014-02-06 19:45:10 -08001560
Geremy Condra36bd3652014-02-06 19:45:10 -08001561 source_boot = common.GetBootableImage(
Tao Bao481bab82017-12-21 11:23:09 -08001562 "/tmp/boot.img", "boot.img", OPTIONS.source_tmp, "BOOT", source_info)
Geremy Condra36bd3652014-02-06 19:45:10 -08001563 target_boot = common.GetBootableImage(
Tao Bao481bab82017-12-21 11:23:09 -08001564 "/tmp/boot.img", "boot.img", OPTIONS.target_tmp, "BOOT", target_info)
Geremy Condra36bd3652014-02-06 19:45:10 -08001565 updating_boot = (not OPTIONS.two_step and
1566 (source_boot.data != target_boot.data))
1567
Geremy Condra36bd3652014-02-06 19:45:10 -08001568 target_recovery = common.GetBootableImage(
1569 "/tmp/recovery.img", "recovery.img", OPTIONS.target_tmp, "RECOVERY")
Geremy Condra36bd3652014-02-06 19:45:10 -08001570
Tianjie Xuf67dd802019-05-20 17:50:36 -07001571 block_diff_dict = GetBlockDifferences(target_zip=target_zip,
1572 source_zip=source_zip,
1573 target_info=target_info,
1574 source_info=source_info,
1575 device_specific=device_specific)
Geremy Condra36bd3652014-02-06 19:45:10 -08001576
Yifan Hong9276cf02019-08-21 16:37:04 -07001577 CheckVintfIfTrebleEnabled(OPTIONS.target_tmp, target_info)
Tao Baobcd1d162017-08-26 13:10:26 -07001578
Tao Bao481bab82017-12-21 11:23:09 -08001579 # Assertions (e.g. device properties check).
1580 target_info.WriteDeviceAssertions(script, OPTIONS.oem_no_mount)
Geremy Condra36bd3652014-02-06 19:45:10 -08001581 device_specific.IncrementalOTA_Assertions()
1582
1583 # Two-step incremental package strategy (in chronological order,
1584 # which is *not* the order in which the generated script has
1585 # things):
1586 #
1587 # if stage is not "2/3" or "3/3":
1588 # do verification on current system
1589 # write recovery image to boot partition
1590 # set stage to "2/3"
1591 # reboot to boot partition and restart recovery
1592 # else if stage is "2/3":
1593 # write recovery image to recovery partition
1594 # set stage to "3/3"
1595 # reboot to recovery partition and restart recovery
1596 # else:
1597 # (stage must be "3/3")
1598 # perform update:
1599 # patch system files, etc.
1600 # force full install of new boot image
1601 # set up system to update recovery partition on first boot
Dan Albert8b72aef2015-03-23 19:13:21 -07001602 # complete script normally
1603 # (allow recovery to mark itself finished and reboot)
Geremy Condra36bd3652014-02-06 19:45:10 -08001604
1605 if OPTIONS.two_step:
Tao Bao481bab82017-12-21 11:23:09 -08001606 if not source_info.get("multistage_support"):
Geremy Condra36bd3652014-02-06 19:45:10 -08001607 assert False, "two-step packages not supported by this build"
Tao Bao24604cc2018-02-01 16:25:44 -08001608 fs = source_info["fstab"]["/misc"]
Geremy Condra36bd3652014-02-06 19:45:10 -08001609 assert fs.fs_type.upper() == "EMMC", \
1610 "two-step packages only supported on devices with EMMC /misc partitions"
Tao Bao481bab82017-12-21 11:23:09 -08001611 bcb_dev = {"bcb_dev" : fs.device}
Geremy Condra36bd3652014-02-06 19:45:10 -08001612 common.ZipWriteStr(output_zip, "recovery.img", target_recovery.data)
1613 script.AppendExtra("""
Michael Rungefb8886d2014-10-23 13:51:04 -07001614if get_stage("%(bcb_dev)s") == "2/3" then
Geremy Condra36bd3652014-02-06 19:45:10 -08001615""" % bcb_dev)
Tao Baod42e97e2016-11-30 12:11:57 -08001616
1617 # Stage 2/3: Write recovery image to /recovery (currently running /boot).
1618 script.Comment("Stage 2/3")
Dan Albert8b72aef2015-03-23 19:13:21 -07001619 script.AppendExtra("sleep(20);\n")
Geremy Condra36bd3652014-02-06 19:45:10 -08001620 script.WriteRawImage("/recovery", "recovery.img")
1621 script.AppendExtra("""
1622set_stage("%(bcb_dev)s", "3/3");
1623reboot_now("%(bcb_dev)s", "recovery");
Michael Rungefb8886d2014-10-23 13:51:04 -07001624else if get_stage("%(bcb_dev)s") != "3/3" then
Geremy Condra36bd3652014-02-06 19:45:10 -08001625""" % bcb_dev)
1626
Tao Baod42e97e2016-11-30 12:11:57 -08001627 # Stage 1/3: (a) Verify the current system.
1628 script.Comment("Stage 1/3")
1629
Tao Bao6c55a8a2015-04-08 15:30:27 -07001630 # Dump fingerprints
Tao Bao481bab82017-12-21 11:23:09 -08001631 script.Print("Source: {}".format(source_info.fingerprint))
1632 script.Print("Target: {}".format(target_info.fingerprint))
Tao Bao6c55a8a2015-04-08 15:30:27 -07001633
Geremy Condra36bd3652014-02-06 19:45:10 -08001634 script.Print("Verifying current system...")
1635
1636 device_specific.IncrementalOTA_VerifyBegin()
1637
Tao Bao481bab82017-12-21 11:23:09 -08001638 WriteFingerprintAssertion(script, target_info, source_info)
Geremy Condra36bd3652014-02-06 19:45:10 -08001639
Tao Baod8d14be2016-02-04 14:26:02 -08001640 # Check the required cache size (i.e. stashed blocks).
Tianjie Xuf67dd802019-05-20 17:50:36 -07001641 required_cache_sizes = [diff.required_cache for diff in
1642 block_diff_dict.values()]
Geremy Condra36bd3652014-02-06 19:45:10 -08001643 if updating_boot:
Tao Bao481bab82017-12-21 11:23:09 -08001644 boot_type, boot_device = common.GetTypeAndDevice("/boot", source_info)
Geremy Condra36bd3652014-02-06 19:45:10 -08001645 d = common.Difference(target_boot, source_boot)
1646 _, _, d = d.ComputePatch()
Doug Zongkerf8340082014-08-05 10:39:37 -07001647 if d is None:
1648 include_full_boot = True
1649 common.ZipWriteStr(output_zip, "boot.img", target_boot.data)
1650 else:
1651 include_full_boot = False
Geremy Condra36bd3652014-02-06 19:45:10 -08001652
Tao Bao32fcdab2018-10-12 10:30:39 -07001653 logger.info(
1654 "boot target: %d source: %d diff: %d", target_boot.size,
1655 source_boot.size, len(d))
Geremy Condra36bd3652014-02-06 19:45:10 -08001656
Tao Bao51216552018-08-26 11:53:15 -07001657 common.ZipWriteStr(output_zip, "boot.img.p", d)
Geremy Condra36bd3652014-02-06 19:45:10 -08001658
Tao Bao51216552018-08-26 11:53:15 -07001659 script.PatchPartitionCheck(
1660 "{}:{}:{}:{}".format(
1661 boot_type, boot_device, target_boot.size, target_boot.sha1),
1662 "{}:{}:{}:{}".format(
1663 boot_type, boot_device, source_boot.size, source_boot.sha1))
1664
Tianjie Xuf67dd802019-05-20 17:50:36 -07001665 required_cache_sizes.append(target_boot.size)
Tao Baod8d14be2016-02-04 14:26:02 -08001666
Tianjie Xuf67dd802019-05-20 17:50:36 -07001667 if required_cache_sizes:
1668 script.CacheFreeSpaceCheck(max(required_cache_sizes))
1669
1670 # Verify the existing partitions.
1671 for diff in block_diff_dict.values():
1672 diff.WriteVerifyScript(script, touched_blocks_only=True)
Geremy Condra36bd3652014-02-06 19:45:10 -08001673
1674 device_specific.IncrementalOTA_VerifyEnd()
1675
1676 if OPTIONS.two_step:
Tao Baod42e97e2016-11-30 12:11:57 -08001677 # Stage 1/3: (b) Write recovery image to /boot.
1678 _WriteRecoveryImageToBoot(script, output_zip)
1679
Geremy Condra36bd3652014-02-06 19:45:10 -08001680 script.AppendExtra("""
1681set_stage("%(bcb_dev)s", "2/3");
1682reboot_now("%(bcb_dev)s", "");
1683else
1684""" % bcb_dev)
1685
Tao Baod42e97e2016-11-30 12:11:57 -08001686 # Stage 3/3: Make changes.
1687 script.Comment("Stage 3/3")
1688
Geremy Condra36bd3652014-02-06 19:45:10 -08001689 script.Comment("---- start making changes here ----")
1690
1691 device_specific.IncrementalOTA_InstallBegin()
1692
Tianjie Xuf67dd802019-05-20 17:50:36 -07001693 progress_dict = {partition: 0.1 for partition in block_diff_dict}
1694 progress_dict["system"] = 1 - len(block_diff_dict) * 0.1
Yifan Hong10c530d2018-12-27 17:34:18 -08001695
1696 if OPTIONS.source_info_dict.get("use_dynamic_partitions") == "true":
1697 if OPTIONS.target_info_dict.get("use_dynamic_partitions") != "true":
1698 raise RuntimeError(
1699 "can't generate incremental that disables dynamic partitions")
1700 dynamic_partitions_diff = common.DynamicPartitionsDifference(
1701 info_dict=OPTIONS.target_info_dict,
1702 source_info_dict=OPTIONS.source_info_dict,
Tianjie Xuf67dd802019-05-20 17:50:36 -07001703 block_diffs=block_diff_dict.values(),
Yifan Hong10c530d2018-12-27 17:34:18 -08001704 progress_dict=progress_dict)
1705 dynamic_partitions_diff.WriteScript(
1706 script, output_zip, write_verify_script=OPTIONS.verify)
1707 else:
Tianjie Xuf67dd802019-05-20 17:50:36 -07001708 for block_diff in block_diff_dict.values():
Yifan Hong10c530d2018-12-27 17:34:18 -08001709 block_diff.WriteScript(script, output_zip,
1710 progress=progress_dict.get(block_diff.partition),
1711 write_verify_script=OPTIONS.verify)
Geremy Condra36bd3652014-02-06 19:45:10 -08001712
1713 if OPTIONS.two_step:
1714 common.ZipWriteStr(output_zip, "boot.img", target_boot.data)
1715 script.WriteRawImage("/boot", "boot.img")
Tao Bao32fcdab2018-10-12 10:30:39 -07001716 logger.info("writing full boot image (forced by two-step mode)")
Geremy Condra36bd3652014-02-06 19:45:10 -08001717
1718 if not OPTIONS.two_step:
1719 if updating_boot:
Doug Zongkerf8340082014-08-05 10:39:37 -07001720 if include_full_boot:
Tao Bao32fcdab2018-10-12 10:30:39 -07001721 logger.info("boot image changed; including full.")
Doug Zongkerf8340082014-08-05 10:39:37 -07001722 script.Print("Installing boot image...")
1723 script.WriteRawImage("/boot", "boot.img")
1724 else:
1725 # Produce the boot image by applying a patch to the current
1726 # contents of the boot partition, and write it back to the
1727 # partition.
Tao Bao32fcdab2018-10-12 10:30:39 -07001728 logger.info("boot image changed; including patch.")
Doug Zongkerf8340082014-08-05 10:39:37 -07001729 script.Print("Patching boot image...")
1730 script.ShowProgress(0.1, 10)
Tao Bao51216552018-08-26 11:53:15 -07001731 script.PatchPartition(
1732 '{}:{}:{}:{}'.format(
1733 boot_type, boot_device, target_boot.size, target_boot.sha1),
1734 '{}:{}:{}:{}'.format(
1735 boot_type, boot_device, source_boot.size, source_boot.sha1),
1736 'boot.img.p')
Geremy Condra36bd3652014-02-06 19:45:10 -08001737 else:
Tao Bao32fcdab2018-10-12 10:30:39 -07001738 logger.info("boot image unchanged; skipping.")
Geremy Condra36bd3652014-02-06 19:45:10 -08001739
1740 # Do device-specific installation (eg, write radio image).
1741 device_specific.IncrementalOTA_InstallEnd()
1742
1743 if OPTIONS.extra_script is not None:
1744 script.AppendExtra(OPTIONS.extra_script)
1745
Doug Zongker922206e2014-03-04 13:16:24 -08001746 if OPTIONS.wipe_user_data:
1747 script.Print("Erasing user data...")
1748 script.FormatPartition("/data")
1749
Geremy Condra36bd3652014-02-06 19:45:10 -08001750 if OPTIONS.two_step:
1751 script.AppendExtra("""
1752set_stage("%(bcb_dev)s", "");
1753endif;
1754endif;
1755""" % bcb_dev)
1756
1757 script.SetProgress(1)
Tao Bao4996cf02016-03-08 17:53:39 -08001758 # For downgrade OTAs, we prefer to use the update-binary in the source
1759 # build that is actually newer than the one in the target build.
1760 if OPTIONS.downgrade:
1761 script.AddToZip(source_zip, output_zip, input_path=OPTIONS.updater_binary)
1762 else:
1763 script.AddToZip(target_zip, output_zip, input_path=OPTIONS.updater_binary)
Tao Baod8d14be2016-02-04 14:26:02 -08001764 metadata["ota-required-cache"] = str(script.required_cache)
Tao Bao491d7e22018-02-21 13:17:22 -08001765
1766 # We haven't written the metadata entry yet, which will be handled in
1767 # FinalizeMetadata().
1768 common.ZipClose(output_zip)
1769
1770 # Sign the generated zip package unless no_signing is specified.
1771 needed_property_files = (
1772 NonAbOtaPropertyFiles(),
1773 )
1774 FinalizeMetadata(metadata, staging_file, output_file, needed_property_files)
Geremy Condra36bd3652014-02-06 19:45:10 -08001775
Doug Zongker32b527d2014-03-04 10:03:02 -08001776
Tao Bao15a146a2018-02-21 16:06:59 -08001777def GetTargetFilesZipForSecondaryImages(input_file, skip_postinstall=False):
Tao Baof7140c02018-01-30 17:09:24 -08001778 """Returns a target-files.zip file for generating secondary payload.
1779
1780 Although the original target-files.zip already contains secondary slot
1781 images (i.e. IMAGES/system_other.img), we need to rename the files to the
1782 ones without _other suffix. Note that we cannot instead modify the names in
1783 META/ab_partitions.txt, because there are no matching partitions on device.
1784
1785 For the partitions that don't have secondary images, the ones for primary
1786 slot will be used. This is to ensure that we always have valid boot, vbmeta,
1787 bootloader images in the inactive slot.
1788
1789 Args:
1790 input_file: The input target-files.zip file.
Tao Bao15a146a2018-02-21 16:06:59 -08001791 skip_postinstall: Whether to skip copying the postinstall config file.
Tao Baof7140c02018-01-30 17:09:24 -08001792
1793 Returns:
1794 The filename of the target-files.zip for generating secondary payload.
1795 """
Tianjie Xu1c808002019-09-11 00:29:26 -07001796
1797 def GetInfoForSecondaryImages(info_file):
1798 """Updates info file for secondary payload generation.
1799
1800 Scan each line in the info file, and remove the unwanted partitions from
1801 the dynamic partition list in the related properties. e.g.
1802 "super_google_dynamic_partitions_partition_list=system vendor product"
1803 will become "super_google_dynamic_partitions_partition_list=system".
1804
1805 Args:
1806 info_file: The input info file. e.g. misc_info.txt.
1807
1808 Returns:
1809 A string of the updated info content.
1810 """
1811
1812 output_list = []
1813 with open(info_file) as f:
1814 lines = f.read().splitlines()
1815
1816 # The suffix in partition_list variables that follows the name of the
1817 # partition group.
1818 LIST_SUFFIX = 'partition_list'
1819 for line in lines:
1820 if line.startswith('#') or '=' not in line:
1821 output_list.append(line)
1822 continue
1823 key, value = line.strip().split('=', 1)
1824 if key == 'dynamic_partition_list' or key.endswith(LIST_SUFFIX):
1825 partitions = value.split()
1826 partitions = [partition for partition in partitions if partition
1827 not in SECONDARY_IMAGES_SKIP_PARTITIONS]
1828 output_list.append('{}={}'.format(key, ' '.join(partitions)))
1829 else:
1830 output_list.append(line)
1831 return '\n'.join(output_list)
1832
Tao Baof7140c02018-01-30 17:09:24 -08001833 target_file = common.MakeTempFile(prefix="targetfiles-", suffix=".zip")
1834 target_zip = zipfile.ZipFile(target_file, 'w', allowZip64=True)
1835
Tao Baodba59ee2018-01-09 13:21:02 -08001836 with zipfile.ZipFile(input_file, 'r') as input_zip:
1837 infolist = input_zip.infolist()
Tao Bao12489802018-07-12 14:47:38 -07001838
Tao Bao0ff15de2019-03-20 11:26:06 -07001839 input_tmp = common.UnzipTemp(input_file, UNZIP_PATTERN)
Tao Baodba59ee2018-01-09 13:21:02 -08001840 for info in infolist:
Tao Baof7140c02018-01-30 17:09:24 -08001841 unzipped_file = os.path.join(input_tmp, *info.filename.split('/'))
1842 if info.filename == 'IMAGES/system_other.img':
1843 common.ZipWrite(target_zip, unzipped_file, arcname='IMAGES/system.img')
1844
1845 # Primary images and friends need to be skipped explicitly.
1846 elif info.filename in ('IMAGES/system.img',
1847 'IMAGES/system.map'):
1848 pass
Tianjie Xu1c808002019-09-11 00:29:26 -07001849 # Images like vendor and product are not needed in the secondary payload.
1850 elif info.filename in ['IMAGES/{}.img'.format(partition) for partition in
1851 SECONDARY_IMAGES_SKIP_PARTITIONS]:
1852 pass
Tao Baof7140c02018-01-30 17:09:24 -08001853
Tao Bao15a146a2018-02-21 16:06:59 -08001854 # Skip copying the postinstall config if requested.
1855 elif skip_postinstall and info.filename == POSTINSTALL_CONFIG:
1856 pass
1857
Tianjie Xu1c808002019-09-11 00:29:26 -07001858 elif info.filename.startswith('META/'):
1859 # Remove the unnecessary partitions for secondary images from the
1860 # ab_partitions file.
1861 if info.filename == AB_PARTITIONS:
1862 with open(unzipped_file) as f:
1863 partition_list = f.read().splitlines()
1864 partition_list = [partition for partition in partition_list if partition
1865 and partition not in SECONDARY_IMAGES_SKIP_PARTITIONS]
1866 common.ZipWriteStr(target_zip, info.filename, '\n'.join(partition_list))
1867 # Remove the unnecessary partitions from the dynamic partitions list.
1868 elif (info.filename == 'META/misc_info.txt' or
1869 info.filename == DYNAMIC_PARTITION_INFO):
1870 modified_info = GetInfoForSecondaryImages(unzipped_file)
1871 common.ZipWriteStr(target_zip, info.filename, modified_info)
1872 else:
1873 common.ZipWrite(target_zip, unzipped_file, arcname=info.filename)
1874 elif info.filename.startswith(('IMAGES/', 'RADIO/')):
Tao Baof7140c02018-01-30 17:09:24 -08001875 common.ZipWrite(target_zip, unzipped_file, arcname=info.filename)
1876
Tao Baof7140c02018-01-30 17:09:24 -08001877 common.ZipClose(target_zip)
1878
1879 return target_file
1880
1881
Tao Bao15a146a2018-02-21 16:06:59 -08001882def GetTargetFilesZipWithoutPostinstallConfig(input_file):
1883 """Returns a target-files.zip that's not containing postinstall_config.txt.
1884
1885 This allows brillo_update_payload script to skip writing all the postinstall
1886 hooks in the generated payload. The input target-files.zip file will be
1887 duplicated, with 'META/postinstall_config.txt' skipped. If input_file doesn't
1888 contain the postinstall_config.txt entry, the input file will be returned.
1889
1890 Args:
1891 input_file: The input target-files.zip filename.
1892
1893 Returns:
1894 The filename of target-files.zip that doesn't contain postinstall config.
1895 """
1896 # We should only make a copy if postinstall_config entry exists.
1897 with zipfile.ZipFile(input_file, 'r') as input_zip:
1898 if POSTINSTALL_CONFIG not in input_zip.namelist():
1899 return input_file
1900
1901 target_file = common.MakeTempFile(prefix="targetfiles-", suffix=".zip")
1902 shutil.copyfile(input_file, target_file)
1903 common.ZipDelete(target_file, POSTINSTALL_CONFIG)
1904 return target_file
1905
1906
Yifan Hong50e79542018-11-08 17:44:12 -08001907def GetTargetFilesZipForRetrofitDynamicPartitions(input_file,
Yifan Hongb433eba2019-03-06 12:42:53 -08001908 super_block_devices,
1909 dynamic_partition_list):
Yifan Hong50e79542018-11-08 17:44:12 -08001910 """Returns a target-files.zip for retrofitting dynamic partitions.
1911
1912 This allows brillo_update_payload to generate an OTA based on the exact
1913 bits on the block devices. Postinstall is disabled.
1914
1915 Args:
1916 input_file: The input target-files.zip filename.
1917 super_block_devices: The list of super block devices
Yifan Hongb433eba2019-03-06 12:42:53 -08001918 dynamic_partition_list: The list of dynamic partitions
Yifan Hong50e79542018-11-08 17:44:12 -08001919
1920 Returns:
1921 The filename of target-files.zip with *.img replaced with super_*.img for
1922 each block device in super_block_devices.
1923 """
1924 assert super_block_devices, "No super_block_devices are specified."
1925
1926 replace = {'OTA/super_{}.img'.format(dev): 'IMAGES/{}.img'.format(dev)
Tao Bao03fecb62018-11-28 10:59:23 -08001927 for dev in super_block_devices}
Yifan Hong50e79542018-11-08 17:44:12 -08001928
1929 target_file = common.MakeTempFile(prefix="targetfiles-", suffix=".zip")
1930 shutil.copyfile(input_file, target_file)
1931
Tao Baoa3705452019-06-24 15:33:41 -07001932 with zipfile.ZipFile(input_file) as input_zip:
Yifan Hong50e79542018-11-08 17:44:12 -08001933 namelist = input_zip.namelist()
1934
Yifan Hongb433eba2019-03-06 12:42:53 -08001935 input_tmp = common.UnzipTemp(input_file, RETROFIT_DAP_UNZIP_PATTERN)
1936
1937 # Remove partitions from META/ab_partitions.txt that is in
1938 # dynamic_partition_list but not in super_block_devices so that
1939 # brillo_update_payload won't generate update for those logical partitions.
1940 ab_partitions_file = os.path.join(input_tmp, *AB_PARTITIONS.split('/'))
1941 with open(ab_partitions_file) as f:
1942 ab_partitions_lines = f.readlines()
1943 ab_partitions = [line.strip() for line in ab_partitions_lines]
1944 # Assert that all super_block_devices are in ab_partitions
1945 super_device_not_updated = [partition for partition in super_block_devices
1946 if partition not in ab_partitions]
1947 assert not super_device_not_updated, \
1948 "{} is in super_block_devices but not in {}".format(
1949 super_device_not_updated, AB_PARTITIONS)
1950 # ab_partitions -= (dynamic_partition_list - super_block_devices)
1951 new_ab_partitions = common.MakeTempFile(prefix="ab_partitions", suffix=".txt")
1952 with open(new_ab_partitions, 'w') as f:
1953 for partition in ab_partitions:
1954 if (partition in dynamic_partition_list and
1955 partition not in super_block_devices):
Tao Bao59cf0c52019-06-25 10:04:24 -07001956 logger.info("Dropping %s from ab_partitions.txt", partition)
1957 continue
Yifan Hongb433eba2019-03-06 12:42:53 -08001958 f.write(partition + "\n")
1959 to_delete = [AB_PARTITIONS]
1960
Yifan Hong50e79542018-11-08 17:44:12 -08001961 # Always skip postinstall for a retrofit update.
Yifan Hongb433eba2019-03-06 12:42:53 -08001962 to_delete += [POSTINSTALL_CONFIG]
Yifan Hong50e79542018-11-08 17:44:12 -08001963
1964 # Delete dynamic_partitions_info.txt so that brillo_update_payload thinks this
1965 # is a regular update on devices without dynamic partitions support.
1966 to_delete += [DYNAMIC_PARTITION_INFO]
1967
Tao Bao03fecb62018-11-28 10:59:23 -08001968 # Remove the existing partition images as well as the map files.
Tao Bao59cf0c52019-06-25 10:04:24 -07001969 to_delete += list(replace.values())
Tao Bao03fecb62018-11-28 10:59:23 -08001970 to_delete += ['IMAGES/{}.map'.format(dev) for dev in super_block_devices]
Yifan Hong50e79542018-11-08 17:44:12 -08001971
1972 common.ZipDelete(target_file, to_delete)
1973
Yifan Hong50e79542018-11-08 17:44:12 -08001974 target_zip = zipfile.ZipFile(target_file, 'a', allowZip64=True)
1975
1976 # Write super_{foo}.img as {foo}.img.
1977 for src, dst in replace.items():
1978 assert src in namelist, \
Tao Bao59cf0c52019-06-25 10:04:24 -07001979 'Missing {} in {}; {} cannot be written'.format(src, input_file, dst)
Yifan Hong50e79542018-11-08 17:44:12 -08001980 unzipped_file = os.path.join(input_tmp, *src.split('/'))
1981 common.ZipWrite(target_zip, unzipped_file, arcname=dst)
1982
Yifan Hongb433eba2019-03-06 12:42:53 -08001983 # Write new ab_partitions.txt file
1984 common.ZipWrite(target_zip, new_ab_partitions, arcname=AB_PARTITIONS)
1985
Yifan Hong50e79542018-11-08 17:44:12 -08001986 common.ZipClose(target_zip)
1987
1988 return target_file
1989
1990
Tao Baof0c4aa22018-04-30 20:29:30 -07001991def GenerateAbOtaPackage(target_file, output_file, source_file=None):
Tao Baofe5b69a2018-03-02 09:47:43 -08001992 """Generates an Android OTA package that has A/B update payload."""
Tao Baodea0f8b2016-06-20 17:55:06 -07001993 # Stage the output zip package for package signing.
Tao Bao491d7e22018-02-21 13:17:22 -08001994 if not OPTIONS.no_signing:
1995 staging_file = common.MakeTempFile(suffix='.zip')
1996 else:
1997 staging_file = output_file
Tao Baoa652c002018-03-01 19:31:38 -08001998 output_zip = zipfile.ZipFile(staging_file, "w",
Tao Baoc098e9e2016-01-07 13:03:56 -08001999 compression=zipfile.ZIP_DEFLATED)
2000
Tao Bao481bab82017-12-21 11:23:09 -08002001 if source_file is not None:
2002 target_info = BuildInfo(OPTIONS.target_info_dict, OPTIONS.oem_dicts)
2003 source_info = BuildInfo(OPTIONS.source_info_dict, OPTIONS.oem_dicts)
2004 else:
2005 target_info = BuildInfo(OPTIONS.info_dict, OPTIONS.oem_dicts)
2006 source_info = None
Tao Baoc098e9e2016-01-07 13:03:56 -08002007
Tao Bao481bab82017-12-21 11:23:09 -08002008 # Metadata to comply with Android OTA package format.
Tao Baodf3a48b2018-01-10 16:30:43 -08002009 metadata = GetPackageMetadata(target_info, source_info)
Tao Baob31892e2017-02-07 11:21:17 -08002010
Yifan Hong50e79542018-11-08 17:44:12 -08002011 if OPTIONS.retrofit_dynamic_partitions:
2012 target_file = GetTargetFilesZipForRetrofitDynamicPartitions(
Yifan Hongb433eba2019-03-06 12:42:53 -08002013 target_file, target_info.get("super_block_devices").strip().split(),
2014 target_info.get("dynamic_partition_list").strip().split())
Yifan Hong50e79542018-11-08 17:44:12 -08002015 elif OPTIONS.skip_postinstall:
Tao Bao15a146a2018-02-21 16:06:59 -08002016 target_file = GetTargetFilesZipWithoutPostinstallConfig(target_file)
2017
Tao Bao40b18822018-01-30 18:19:04 -08002018 # Generate payload.
2019 payload = Payload()
2020
2021 # Enforce a max timestamp this payload can be applied on top of.
Tao Baoff1b86e2017-10-03 14:17:57 -07002022 if OPTIONS.downgrade:
Tao Bao2a12ed72018-01-22 11:35:00 -08002023 max_timestamp = source_info.GetBuildProp("ro.build.date.utc")
Tao Baoff1b86e2017-10-03 14:17:57 -07002024 else:
2025 max_timestamp = metadata["post-timestamp"]
Tao Bao40b18822018-01-30 18:19:04 -08002026 additional_args = ["--max_timestamp", max_timestamp]
Tao Baoc098e9e2016-01-07 13:03:56 -08002027
Tao Bao40b18822018-01-30 18:19:04 -08002028 payload.Generate(target_file, source_file, additional_args)
Tao Baoc098e9e2016-01-07 13:03:56 -08002029
Tao Bao40b18822018-01-30 18:19:04 -08002030 # Sign the payload.
Tao Baof7140c02018-01-30 17:09:24 -08002031 payload_signer = PayloadSigner()
2032 payload.Sign(payload_signer)
Tao Baoc098e9e2016-01-07 13:03:56 -08002033
Tao Bao40b18822018-01-30 18:19:04 -08002034 # Write the payload into output zip.
2035 payload.WriteToZip(output_zip)
Tao Baoc098e9e2016-01-07 13:03:56 -08002036
Tao Baof7140c02018-01-30 17:09:24 -08002037 # Generate and include the secondary payload that installs secondary images
2038 # (e.g. system_other.img).
2039 if OPTIONS.include_secondary:
2040 # We always include a full payload for the secondary slot, even when
2041 # building an incremental OTA. See the comments for "--include_secondary".
Tao Bao15a146a2018-02-21 16:06:59 -08002042 secondary_target_file = GetTargetFilesZipForSecondaryImages(
2043 target_file, OPTIONS.skip_postinstall)
Tao Bao667ff572018-02-10 00:02:40 -08002044 secondary_payload = Payload(secondary=True)
Tao Baodb1fe412018-02-09 23:15:05 -08002045 secondary_payload.Generate(secondary_target_file,
2046 additional_args=additional_args)
Tao Baof7140c02018-01-30 17:09:24 -08002047 secondary_payload.Sign(payload_signer)
Tao Bao667ff572018-02-10 00:02:40 -08002048 secondary_payload.WriteToZip(output_zip)
Tao Baof7140c02018-01-30 17:09:24 -08002049
Tianjie Xucfa86222016-03-07 16:31:19 -08002050 # If dm-verity is supported for the device, copy contents of care_map
2051 # into A/B OTA package.
Tao Bao21803d32017-04-19 10:16:09 -07002052 target_zip = zipfile.ZipFile(target_file, "r")
Tao Bao481bab82017-12-21 11:23:09 -08002053 if (target_info.get("verity") == "true" or
2054 target_info.get("avb_enable") == "true"):
Tianjie Xu4c05f4a2018-09-14 16:24:41 -07002055 care_map_list = [x for x in ["care_map.pb", "care_map.txt"] if
2056 "META/" + x in target_zip.namelist()]
2057
2058 # Adds care_map if either the protobuf format or the plain text one exists.
2059 if care_map_list:
2060 care_map_name = care_map_list[0]
2061 care_map_data = target_zip.read("META/" + care_map_name)
2062 # In order to support streaming, care_map needs to be packed as
Tao Bao40b18822018-01-30 18:19:04 -08002063 # ZIP_STORED.
Tianjie Xu4c05f4a2018-09-14 16:24:41 -07002064 common.ZipWriteStr(output_zip, care_map_name, care_map_data,
Tao Bao481bab82017-12-21 11:23:09 -08002065 compress_type=zipfile.ZIP_STORED)
Tianjie Xucfa86222016-03-07 16:31:19 -08002066 else:
Tao Bao32fcdab2018-10-12 10:30:39 -07002067 logger.warning("Cannot find care map file in target_file package")
Tao Bao21803d32017-04-19 10:16:09 -07002068
Tao Bao21803d32017-04-19 10:16:09 -07002069 common.ZipClose(target_zip)
Tianjie Xucfa86222016-03-07 16:31:19 -08002070
Yifan Hong9276cf02019-08-21 16:37:04 -07002071 CheckVintfIfTrebleEnabled(target_file, target_info)
2072
Tao Baofe5b69a2018-03-02 09:47:43 -08002073 # We haven't written the metadata entry yet, which will be handled in
2074 # FinalizeMetadata().
Tao Baoc96316c2017-01-24 22:10:49 -08002075 common.ZipClose(output_zip)
2076
Tao Bao85f16982018-03-08 16:28:33 -08002077 # AbOtaPropertyFiles intends to replace StreamingPropertyFiles, as it covers
2078 # all the info of the latter. However, system updaters and OTA servers need to
2079 # take time to switch to the new flag. We keep both of the flags for
2080 # P-timeframe, and will remove StreamingPropertyFiles in later release.
Tao Baod3fc38a2018-03-08 16:09:01 -08002081 needed_property_files = (
Tao Bao85f16982018-03-08 16:28:33 -08002082 AbOtaPropertyFiles(),
Tao Baod3fc38a2018-03-08 16:09:01 -08002083 StreamingPropertyFiles(),
2084 )
2085 FinalizeMetadata(metadata, staging_file, output_file, needed_property_files)
Tao Baoc96316c2017-01-24 22:10:49 -08002086
Tao Baoc098e9e2016-01-07 13:03:56 -08002087
Tao Baof0c4aa22018-04-30 20:29:30 -07002088def GenerateNonAbOtaPackage(target_file, output_file, source_file=None):
2089 """Generates a non-A/B OTA package."""
2090 # Sanity check the loaded info dicts first.
2091 if OPTIONS.info_dict.get("no_recovery") == "true":
2092 raise common.ExternalError(
2093 "--- target build has specified no recovery ---")
2094
2095 # Non-A/B OTAs rely on /cache partition to store temporary files.
2096 cache_size = OPTIONS.info_dict.get("cache_size")
2097 if cache_size is None:
2098 logger.warning("--- can't determine the cache partition size ---")
2099 OPTIONS.cache_size = cache_size
2100
2101 if OPTIONS.extra_script is not None:
2102 with open(OPTIONS.extra_script) as fp:
2103 OPTIONS.extra_script = fp.read()
2104
2105 if OPTIONS.extracted_input is not None:
2106 OPTIONS.input_tmp = OPTIONS.extracted_input
2107 else:
2108 logger.info("unzipping target target-files...")
2109 OPTIONS.input_tmp = common.UnzipTemp(target_file, UNZIP_PATTERN)
2110 OPTIONS.target_tmp = OPTIONS.input_tmp
2111
2112 # If the caller explicitly specified the device-specific extensions path via
2113 # -s / --device_specific, use that. Otherwise, use META/releasetools.py if it
2114 # is present in the target target_files. Otherwise, take the path of the file
2115 # from 'tool_extensions' in the info dict and look for that in the local
2116 # filesystem, relative to the current directory.
2117 if OPTIONS.device_specific is None:
2118 from_input = os.path.join(OPTIONS.input_tmp, "META", "releasetools.py")
2119 if os.path.exists(from_input):
2120 logger.info("(using device-specific extensions from target_files)")
2121 OPTIONS.device_specific = from_input
2122 else:
2123 OPTIONS.device_specific = OPTIONS.info_dict.get("tool_extensions")
2124
2125 if OPTIONS.device_specific is not None:
2126 OPTIONS.device_specific = os.path.abspath(OPTIONS.device_specific)
2127
2128 # Generate a full OTA.
2129 if source_file is None:
2130 with zipfile.ZipFile(target_file) as input_zip:
2131 WriteFullOTAPackage(
2132 input_zip,
2133 output_file)
2134
2135 # Generate an incremental OTA.
2136 else:
2137 logger.info("unzipping source target-files...")
2138 OPTIONS.source_tmp = common.UnzipTemp(
2139 OPTIONS.incremental_source, UNZIP_PATTERN)
2140 with zipfile.ZipFile(target_file) as input_zip, \
2141 zipfile.ZipFile(source_file) as source_zip:
2142 WriteBlockIncrementalOTAPackage(
2143 input_zip,
2144 source_zip,
2145 output_file)
2146
2147
Doug Zongkereef39442009-04-02 12:14:19 -07002148def main(argv):
2149
2150 def option_handler(o, a):
Tao Bao4b76a0e2017-10-31 12:13:33 -07002151 if o in ("-k", "--package_key"):
Doug Zongkereef39442009-04-02 12:14:19 -07002152 OPTIONS.package_key = a
Doug Zongkereef39442009-04-02 12:14:19 -07002153 elif o in ("-i", "--incremental_from"):
2154 OPTIONS.incremental_source = a
Tao Bao43078aa2015-04-21 14:32:35 -07002155 elif o == "--full_radio":
2156 OPTIONS.full_radio = True
leozwangaa6c1a12015-08-14 10:57:58 -07002157 elif o == "--full_bootloader":
2158 OPTIONS.full_bootloader = True
Tao Bao337633f2017-12-06 15:20:19 -08002159 elif o == "--wipe_user_data":
Doug Zongkerdbfaae52009-04-21 17:12:54 -07002160 OPTIONS.wipe_user_data = True
Tao Bao5d182562016-02-23 11:38:39 -08002161 elif o == "--downgrade":
2162 OPTIONS.downgrade = True
2163 OPTIONS.wipe_user_data = True
Tao Bao3e6161a2017-02-28 11:48:48 -08002164 elif o == "--override_timestamp":
Tao Baofaa8e0b2018-04-12 14:31:43 -07002165 OPTIONS.downgrade = True
Michael Runge6e836112014-04-15 17:40:21 -07002166 elif o in ("-o", "--oem_settings"):
Alain Vongsouvanh7f804ba2017-02-16 13:06:55 -08002167 OPTIONS.oem_source = a.split(',')
Tao Bao8608cde2016-02-25 19:49:55 -08002168 elif o == "--oem_no_mount":
2169 OPTIONS.oem_no_mount = True
Doug Zongker1c390a22009-05-14 19:06:36 -07002170 elif o in ("-e", "--extra_script"):
2171 OPTIONS.extra_script = a
Martin Blumenstingl374e1142014-05-31 20:42:55 +02002172 elif o in ("-t", "--worker_threads"):
2173 if a.isdigit():
2174 OPTIONS.worker_threads = int(a)
2175 else:
2176 raise ValueError("Cannot parse value %r for option %r - only "
2177 "integers are allowed." % (a, o))
Doug Zongker9b23f2c2013-11-25 14:44:12 -08002178 elif o in ("-2", "--two_step"):
2179 OPTIONS.two_step = True
Tao Baof7140c02018-01-30 17:09:24 -08002180 elif o == "--include_secondary":
2181 OPTIONS.include_secondary = True
Doug Zongker26e66192014-02-20 13:22:07 -08002182 elif o == "--no_signing":
Takeshi Kanemotoe153b342013-11-14 17:20:50 +09002183 OPTIONS.no_signing = True
Dan Albert8b72aef2015-03-23 19:13:21 -07002184 elif o == "--verify":
Michael Runge63f01de2014-10-28 19:24:19 -07002185 OPTIONS.verify = True
Doug Zongker26e66192014-02-20 13:22:07 -08002186 elif o == "--block":
2187 OPTIONS.block_based = True
Doug Zongker25568482014-03-03 10:21:27 -08002188 elif o in ("-b", "--binary"):
2189 OPTIONS.updater_binary = a
Tao Bao8dcf7382015-05-21 14:09:49 -07002190 elif o == "--stash_threshold":
2191 try:
2192 OPTIONS.stash_threshold = float(a)
2193 except ValueError:
2194 raise ValueError("Cannot parse value %r for option %r - expecting "
2195 "a float" % (a, o))
Tao Baod62c6032015-11-30 09:40:20 -08002196 elif o == "--log_diff":
2197 OPTIONS.log_diff = a
Tao Baodea0f8b2016-06-20 17:55:06 -07002198 elif o == "--payload_signer":
2199 OPTIONS.payload_signer = a
Baligh Uddin2abbbd02016-06-22 12:14:16 -07002200 elif o == "--payload_signer_args":
2201 OPTIONS.payload_signer_args = shlex.split(a)
xunchang376cc7c2019-04-08 23:04:58 -07002202 elif o == "--payload_signer_key_size":
2203 OPTIONS.payload_signer_key_size = a
Dan Willemsencea5cd22017-03-21 14:44:27 -07002204 elif o == "--extracted_input_target_files":
2205 OPTIONS.extracted_input = a
Tao Bao15a146a2018-02-21 16:06:59 -08002206 elif o == "--skip_postinstall":
2207 OPTIONS.skip_postinstall = True
Yifan Hong50e79542018-11-08 17:44:12 -08002208 elif o == "--retrofit_dynamic_partitions":
2209 OPTIONS.retrofit_dynamic_partitions = True
xunchangabfa2652019-02-19 16:27:10 -08002210 elif o == "--skip_compatibility_check":
2211 OPTIONS.skip_compatibility_check = True
xunchang1cfe2512019-02-19 14:14:48 -08002212 elif o == "--output_metadata_path":
2213 OPTIONS.output_metadata_path = a
Tianjie Xu1b079832019-08-28 12:19:23 -07002214 elif o == "--disable_fec_computation":
2215 OPTIONS.disable_fec_computation = True
Doug Zongkereef39442009-04-02 12:14:19 -07002216 else:
2217 return False
Doug Zongkerdbfaae52009-04-21 17:12:54 -07002218 return True
Doug Zongkereef39442009-04-02 12:14:19 -07002219
2220 args = common.ParseOptions(argv, __doc__,
Tao Bao337633f2017-12-06 15:20:19 -08002221 extra_opts="b:k:i:d:e:t:2o:",
Dan Albert8b72aef2015-03-23 19:13:21 -07002222 extra_long_opts=[
Dan Albert8b72aef2015-03-23 19:13:21 -07002223 "package_key=",
2224 "incremental_from=",
Tao Bao43078aa2015-04-21 14:32:35 -07002225 "full_radio",
leozwangaa6c1a12015-08-14 10:57:58 -07002226 "full_bootloader",
Dan Albert8b72aef2015-03-23 19:13:21 -07002227 "wipe_user_data",
Tao Bao5d182562016-02-23 11:38:39 -08002228 "downgrade",
Tao Bao3e6161a2017-02-28 11:48:48 -08002229 "override_timestamp",
Dan Albert8b72aef2015-03-23 19:13:21 -07002230 "extra_script=",
2231 "worker_threads=",
Dan Albert8b72aef2015-03-23 19:13:21 -07002232 "two_step",
Tao Baof7140c02018-01-30 17:09:24 -08002233 "include_secondary",
Dan Albert8b72aef2015-03-23 19:13:21 -07002234 "no_signing",
2235 "block",
2236 "binary=",
2237 "oem_settings=",
Tao Bao8608cde2016-02-25 19:49:55 -08002238 "oem_no_mount",
Dan Albert8b72aef2015-03-23 19:13:21 -07002239 "verify",
Tao Bao8dcf7382015-05-21 14:09:49 -07002240 "stash_threshold=",
Tao Baod62c6032015-11-30 09:40:20 -08002241 "log_diff=",
Tao Baodea0f8b2016-06-20 17:55:06 -07002242 "payload_signer=",
Baligh Uddin2abbbd02016-06-22 12:14:16 -07002243 "payload_signer_args=",
xunchang376cc7c2019-04-08 23:04:58 -07002244 "payload_signer_key_size=",
Dan Willemsencea5cd22017-03-21 14:44:27 -07002245 "extracted_input_target_files=",
Tao Bao15a146a2018-02-21 16:06:59 -08002246 "skip_postinstall",
Yifan Hong50e79542018-11-08 17:44:12 -08002247 "retrofit_dynamic_partitions",
xunchangabfa2652019-02-19 16:27:10 -08002248 "skip_compatibility_check",
xunchang1cfe2512019-02-19 14:14:48 -08002249 "output_metadata_path=",
Tianjie Xu1b079832019-08-28 12:19:23 -07002250 "disable_fec_computation",
Dan Albert8b72aef2015-03-23 19:13:21 -07002251 ], extra_option_handler=option_handler)
Doug Zongkereef39442009-04-02 12:14:19 -07002252
2253 if len(args) != 2:
2254 common.Usage(__doc__)
2255 sys.exit(1)
2256
Tao Bao32fcdab2018-10-12 10:30:39 -07002257 common.InitLogging()
2258
Tao Bao5d182562016-02-23 11:38:39 -08002259 if OPTIONS.downgrade:
Tao Bao5d182562016-02-23 11:38:39 -08002260 # We should only allow downgrading incrementals (as opposed to full).
2261 # Otherwise the device may go back from arbitrary build with this full
2262 # OTA package.
2263 if OPTIONS.incremental_source is None:
Elliott Hughesd8a52f92016-06-20 14:35:47 -07002264 raise ValueError("Cannot generate downgradable full OTAs")
Tao Bao5d182562016-02-23 11:38:39 -08002265
Tao Bao2db13852018-01-08 22:28:57 -08002266 # Load the build info dicts from the zip directly or the extracted input
2267 # directory. We don't need to unzip the entire target-files zips, because they
2268 # won't be needed for A/B OTAs (brillo_update_payload does that on its own).
2269 # When loading the info dicts, we don't need to provide the second parameter
2270 # to common.LoadInfoDict(). Specifying the second parameter allows replacing
2271 # some properties with their actual paths, such as 'selinux_fc',
2272 # 'ramdisk_dir', which won't be used during OTA generation.
Dan Willemsencea5cd22017-03-21 14:44:27 -07002273 if OPTIONS.extracted_input is not None:
Tao Bao2db13852018-01-08 22:28:57 -08002274 OPTIONS.info_dict = common.LoadInfoDict(OPTIONS.extracted_input)
Dan Willemsencea5cd22017-03-21 14:44:27 -07002275 else:
Tao Bao2db13852018-01-08 22:28:57 -08002276 with zipfile.ZipFile(args[0], 'r') as input_zip:
2277 OPTIONS.info_dict = common.LoadInfoDict(input_zip)
Tao Baoc098e9e2016-01-07 13:03:56 -08002278
Tao Bao32fcdab2018-10-12 10:30:39 -07002279 logger.info("--- target info ---")
2280 common.DumpInfoDict(OPTIONS.info_dict)
Tao Bao2db13852018-01-08 22:28:57 -08002281
2282 # Load the source build dict if applicable.
2283 if OPTIONS.incremental_source is not None:
2284 OPTIONS.target_info_dict = OPTIONS.info_dict
2285 with zipfile.ZipFile(OPTIONS.incremental_source, 'r') as source_zip:
2286 OPTIONS.source_info_dict = common.LoadInfoDict(source_zip)
2287
Tao Bao32fcdab2018-10-12 10:30:39 -07002288 logger.info("--- source info ---")
2289 common.DumpInfoDict(OPTIONS.source_info_dict)
Tao Bao2db13852018-01-08 22:28:57 -08002290
2291 # Load OEM dicts if provided.
Tao Bao481bab82017-12-21 11:23:09 -08002292 OPTIONS.oem_dicts = _LoadOemDicts(OPTIONS.oem_source)
2293
Yifan Hong50e79542018-11-08 17:44:12 -08002294 # Assume retrofitting dynamic partitions when base build does not set
Yifan Hong50611032018-11-20 14:27:38 -08002295 # use_dynamic_partitions but target build does.
Yifan Hong50e79542018-11-08 17:44:12 -08002296 if (OPTIONS.source_info_dict and
Yifan Hong50611032018-11-20 14:27:38 -08002297 OPTIONS.source_info_dict.get("use_dynamic_partitions") != "true" and
2298 OPTIONS.target_info_dict.get("use_dynamic_partitions") == "true"):
Yifan Hong50e79542018-11-08 17:44:12 -08002299 if OPTIONS.target_info_dict.get("dynamic_partition_retrofit") != "true":
2300 raise common.ExternalError(
2301 "Expect to generate incremental OTA for retrofitting dynamic "
2302 "partitions, but dynamic_partition_retrofit is not set in target "
2303 "build.")
2304 logger.info("Implicitly generating retrofit incremental OTA.")
2305 OPTIONS.retrofit_dynamic_partitions = True
2306
2307 # Skip postinstall for retrofitting dynamic partitions.
2308 if OPTIONS.retrofit_dynamic_partitions:
2309 OPTIONS.skip_postinstall = True
2310
Tao Baoc098e9e2016-01-07 13:03:56 -08002311 ab_update = OPTIONS.info_dict.get("ab_update") == "true"
2312
Christian Oderf63e2cd2017-05-01 22:30:15 +02002313 # Use the default key to sign the package if not specified with package_key.
2314 # package_keys are needed on ab_updates, so always define them if an
2315 # ab_update is getting created.
2316 if not OPTIONS.no_signing or ab_update:
2317 if OPTIONS.package_key is None:
2318 OPTIONS.package_key = OPTIONS.info_dict.get(
2319 "default_system_dev_certificate",
Dan Willemsen0ab1be62019-04-09 21:35:37 -07002320 "build/make/target/product/security/testkey")
Christian Oderf63e2cd2017-05-01 22:30:15 +02002321 # Get signing keys
2322 OPTIONS.key_passwords = common.GetKeyPasswords([OPTIONS.package_key])
2323
Tao Baoc098e9e2016-01-07 13:03:56 -08002324 if ab_update:
Tao Baof0c4aa22018-04-30 20:29:30 -07002325 GenerateAbOtaPackage(
Tao Baoc098e9e2016-01-07 13:03:56 -08002326 target_file=args[0],
2327 output_file=args[1],
2328 source_file=OPTIONS.incremental_source)
2329
Dan Willemsencea5cd22017-03-21 14:44:27 -07002330 else:
Tao Baof0c4aa22018-04-30 20:29:30 -07002331 GenerateNonAbOtaPackage(
2332 target_file=args[0],
2333 output_file=args[1],
2334 source_file=OPTIONS.incremental_source)
Doug Zongkerfdd8e692009-08-03 17:27:48 -07002335
Tao Baof0c4aa22018-04-30 20:29:30 -07002336 # Post OTA generation works.
2337 if OPTIONS.incremental_source is not None and OPTIONS.log_diff:
2338 logger.info("Generating diff logs...")
2339 logger.info("Unzipping target-files for diffing...")
2340 target_dir = common.UnzipTemp(args[0], TARGET_DIFFING_UNZIP_PATTERN)
2341 source_dir = common.UnzipTemp(
2342 OPTIONS.incremental_source, TARGET_DIFFING_UNZIP_PATTERN)
Doug Zongkereb0a78a2014-01-27 10:01:06 -08002343
Tao Baof0c4aa22018-04-30 20:29:30 -07002344 with open(OPTIONS.log_diff, 'w') as out_file:
2345 import target_files_diff
2346 target_files_diff.recursiveDiff(
2347 '', source_dir, target_dir, out_file)
Doug Zongker62d4f182014-08-04 16:06:43 -07002348
Tao Bao32fcdab2018-10-12 10:30:39 -07002349 logger.info("done.")
Doug Zongkereef39442009-04-02 12:14:19 -07002350
2351
2352if __name__ == '__main__':
2353 try:
Ying Wang7e6d4e42010-12-13 16:25:36 -08002354 common.CloseInheritedPipes()
Doug Zongkereef39442009-04-02 12:14:19 -07002355 main(sys.argv[1:])
Tao Bao32fcdab2018-10-12 10:30:39 -07002356 except common.ExternalError:
2357 logger.exception("\n ERROR:\n")
Doug Zongkereef39442009-04-02 12:14:19 -07002358 sys.exit(1)
Doug Zongkerfc44a512014-08-26 13:10:25 -07002359 finally:
2360 common.Cleanup()