Fix caching interface according to vendor feedback.
- Instead of isCachingSupport returning a single boolean, switch to
getNumberOfCacheFilesNeeded returning the number of cache files. This
is to support use cases when driver needs more than one cache file for
each type, or when driver does not need data cache.
- Instead of a separate saveToCache, pass cache info along with
prepareModel_1_2 to save into cache as well as perform compilation.
This is to avoid a potential additional copy of cache files.
Bug: 123780248
Test: VtsHalNeuralnetworksV1_xTargetTest with 1.2 sample driver
Test: VtsHalNeuralnetworksV1_xTargetTest with a test driver that can
read and write cache entries
Change-Id: I921b7b8ccc3c66af19f6589f7213c6870d6f07bf
diff --git a/neuralnetworks/1.2/IDevice.hal b/neuralnetworks/1.2/IDevice.hal
index b9fa388..da9a966 100644
--- a/neuralnetworks/1.2/IDevice.hal
+++ b/neuralnetworks/1.2/IDevice.hal
@@ -113,44 +113,83 @@
generates (ErrorStatus status, vec<bool> supportedOperations);
/**
- * Gets whether the driver supports compilation caching.
+ * Gets the caching requirements of the driver implementation.
*
- * isCachingSupported indicates whether the driver supports compilation caching.
- * Even if so, the driver may still choose not to cache certain compiled models.
+ * There are two types of cache file descriptors provided to the driver: model cache
+ * and data cache.
*
- * If the device reports the caching is not supported, the user may avoid calling
- * IDevice::prepareModelFromCache and IPreparedModel::saveToCache.
+ * The data cache is for caching constant data, possibly including preprocessed
+ * and transformed tensor buffers. Any modification to the data cache should
+ * have no worse effect than generating bad output values at execution time.
+ *
+ * The model cache is for caching security-sensitive data such as compiled
+ * executable machine code in the device's native binary format. A modification
+ * to the model cache may affect the driver's execution behavior, and a malicious
+ * client could make use of this to execute beyond the granted permission. Thus,
+ * the driver must always check whether the model cache is corrupted before
+ * preparing the model from cache.
+ *
+ * getNumberOfCacheFilesNeeded returns how many of each type of cache files the driver
+ * implementation needs to cache a single prepared model. Returning 0 for both types
+ * indicates compilation caching is not supported by this driver. The driver may
+ * still choose not to cache certain compiled models even if it reports that caching
+ * is supported.
+ *
+ * If the device reports that caching is not supported, the user may avoid calling
+ * IDevice::prepareModelFromCache or providing cache file descriptors to
+ * IDevice::prepareModel_1_2.
*
* @return status Error status of the call, must be:
* - NONE if successful
* - DEVICE_UNAVAILABLE if driver is offline or busy
* - GENERAL_FAILURE if there is an unspecified error
- * @return supported A boolean indicating whether the driver supports compilation
- * caching. Even on returning true, the driver may still choose
- * not to cache certain compiled models.
+ * @return numModelCache An unsigned integer indicating how many files for model cache
+ * the driver needs to cache a single prepared model. It must
+ * be less than or equal to Constant::MAX_NUMBER_OF_CACHE_FILES.
+ * @return numDataCache An unsigned integer indicating how many files for data cache
+ * the driver needs to cache a single prepared model. It must
+ * be less than or equal to Constant::MAX_NUMBER_OF_CACHE_FILES.
*/
- isCachingSupported() generates (ErrorStatus status, bool supported);
+ getNumberOfCacheFilesNeeded()
+ generates (ErrorStatus status, uint32_t numModelCache, uint32_t numDataCache);
/**
- * Creates a prepared model for execution.
+ * Asynchronously creates a prepared model for execution and optionally saves it
+ * into cache files.
*
- * prepareModel is used to make any necessary transformations or alternative
+ * prepareModel is used to make any necessary transformations to or alternative
* representations to a model for execution, possibly including
* transformations on the constant data, optimization on the model's graph,
* or compilation into the device's native binary format. The model itself
* is not changed.
*
+ * Optionally, caching information may be provided for the driver to save
+ * the prepared model to cache files for faster model compilation time
+ * when the same model preparation is requested in the future. There are
+ * two types of cache file handles provided to the driver: model cache
+ * and data cache. For more information on the two types of cache handles,
+ * refer to getNumberOfCacheFilesNeeded.
+ *
+ * The file descriptors must be opened with read and write permission. A file may
+ * have any size, and the corresponding file descriptor may have any offset. The
+ * driver must truncate a file to zero size before writing to that file. The file
+ * descriptors may be closed by the client once the asynchronous preparation has
+ * finished. The driver must dup a file descriptor if it wants to get access to
+ * the cache file later.
+ *
* The model is prepared asynchronously with respect to the caller. The
- * prepareModel function must verify the inputs to the prepareModel function
- * are correct. If there is an error, prepareModel must immediately invoke
+ * prepareModel function must verify the inputs to the preparedModel function
+ * related to preparing the model (as opposed to saving the prepared model to
+ * cache) are correct. If there is an error, prepareModel must immediately invoke
* the callback with the appropriate ErrorStatus value and nullptr for the
- * IPreparedModel, then return with the same ErrorStatus. If the inputs to
- * the prepareModel function are valid and there is no error, prepareModel
- * must launch an asynchronous task to prepare the model in the background,
- * and immediately return from prepareModel with ErrorStatus::NONE. If the
- * asynchronous task fails to launch, prepareModel must immediately invoke
- * the callback with ErrorStatus::GENERAL_FAILURE and nullptr for the
- * IPreparedModel, then return with ErrorStatus::GENERAL_FAILURE.
+ * IPreparedModel, then return with the same ErrorStatus. If the inputs to the
+ * prepareModel function that are related to preparing the model are valid and
+ * there is no error, prepareModel must launch an asynchronous task
+ * to prepare the model in the background, and immediately return from
+ * prepareModel with ErrorStatus::NONE. If the asynchronous task fails to launch,
+ * prepareModel must immediately invoke the callback with
+ * ErrorStatus::GENERAL_FAILURE and nullptr for the IPreparedModel, then return
+ * with ErrorStatus::GENERAL_FAILURE.
*
* When the asynchronous task has finished preparing the model, it must
* immediately invoke the callback function provided as an input to
@@ -160,6 +199,14 @@
* the callback object must be invoked with the appropriate ErrorStatus
* value and nullptr for the IPreparedModel.
*
+ * Optionally, the driver may save the prepared model to cache during the
+ * asynchronous preparation. Any error that occurs when saving to cache must
+ * not affect the status of preparing the model. Even if the input arguments
+ * related to the cache may be invalid, or the driver may fail to save to cache,
+ * the prepareModel function must finish preparing the model. The driver
+ * may choose not to save to cache even if the caching information is
+ * provided and valid.
+ *
* The only information that may be unknown to the model at this stage is
* the shape of the tensors, which may only be known at execution time. As
* such, some driver services may return partially prepared models, where
@@ -173,6 +220,26 @@
* @param model The model to be prepared for execution.
* @param preference Indicates the intended execution behavior of a prepared
* model.
+ * @param modelCache A vector of handles with each entry holding exactly one
+ * cache file descriptor for the security-sensitive cache. The length of
+ * the vector must either be 0 indicating that caching information is not provided,
+ * or match the numModelCache returned from getNumberOfCacheFilesNeeded. The cache
+ * handles will be provided in the same order when retrieving the
+ * preparedModel from cache files with prepareModelFromCache.
+ * @param dataCache A vector of handles with each entry holding exactly one
+ * cache file descriptor for the constants' cache. The length of
+ * the vector must either be 0 indicating that caching information is not provided,
+ * or match the numDataCache returned from getNumberOfCacheFilesNeeded. The cache
+ * handles will be provided in the same order when retrieving the
+ * preparedModel from cache files with prepareModelFromCache.
+ * @param token A caching token of length Constant::BYTE_SIZE_OF_CACHE_TOKEN
+ * identifying the prepared model. The same token will be provided when retrieving
+ * the prepared model from the cache files with prepareModelFromCache.
+ * Tokens should be chosen to have a low rate of collision for a particular
+ * application. The driver cannot detect a collision; a collision will result
+ * in a failed execution or in a successful execution that produces incorrect
+ * output values. If both modelCache and dataCache are empty indicating that
+ * caching information is not provided, this token must be ignored.
* @param callback A callback object used to return the error status of
* preparing the model for execution and the prepared model if
* successful, nullptr otherwise. The callback object's notify function
@@ -182,9 +249,12 @@
* - NONE if preparation task is successfully launched
* - DEVICE_UNAVAILABLE if driver is offline or busy
* - GENERAL_FAILURE if there is an unspecified error
- * - INVALID_ARGUMENT if one of the input arguments is invalid
+ * - INVALID_ARGUMENT if one of the input arguments related to preparing the
+ * model is invalid
*/
prepareModel_1_2(Model model, ExecutionPreference preference,
+ vec<handle> modelCache, vec<handle> dataCache,
+ uint8_t[Constant:BYTE_SIZE_OF_CACHE_TOKEN] token,
IPreparedModelCallback callback)
generates (ErrorStatus status);
@@ -192,22 +262,17 @@
* Creates a prepared model from cache files for execution.
*
* prepareModelFromCache is used to retrieve a prepared model directly from
- * cache files to avoid slow model compilation time. There are exactly two
- * cache file descriptors provided to the driver: modelCache and dataCache.
+ * cache files to avoid slow model compilation time. There are
+ * two types of cache file handles provided to the driver: model cache
+ * and data cache. For more information on the two types of cache handles,
+ * refer to getNumberOfCacheFilesNeeded.
*
- * The dataCache is for caching constant data, possibly including preprocessed
- * and transformed tensor buffers. Any modification to the dataCache should
- * have no worse effect than generating bad output values at execution time.
- *
- * The modelCache is for caching security-sensitive data such as compiled
- * executable machine code in the device's native binary format. A modification
- * to the modelCache may affect the driver's execution behavior, and a malicious
- * client could make use of this to execute beyond the granted permission. Thus,
- * the driver must always check whether the modelCache is corrupted before preparing
- * the model from cache.
- *
- * The two file descriptors may be closed by the client once the asynchronous
- * preparation has finished. The driver has to copy all the data it needs.
+ * The file descriptors must be opened with read and write permission. A file may
+ * have any size, and the corresponding file descriptor may have any offset. The
+ * driver must truncate a file to zero size before writing to that file. The file
+ * descriptors may be closed by the client once the asynchronous preparation has
+ * finished. The driver must dup a file descriptor if it wants to get access to
+ * the cache file later.
*
* The model is prepared asynchronously with respect to the caller. The
* prepareModelFromCache function must verify the inputs to the
@@ -241,13 +306,17 @@
* used with different shapes of inputs on different (possibly concurrent)
* executions.
*
- * @param modelCache A handle holding exactly one cache file descriptor for the
- * security-sensitive cache.
- * @param dataCache A handle holding exactly one cache file descriptor for the
- * constants' cache.
+ * @param modelCache A vector of handles with each entry holding exactly one
+ * cache file descriptor for the security-sensitive cache. The length of
+ * the vector must match the numModelCache returned from getNumberOfCacheFilesNeeded.
+ * The cache handles will be provided in the same order as with prepareModel_1_2.
+ * @param dataCache A vector of handles with each entry holding exactly one
+ * cache file descriptor for the constants' cache. The length of the vector
+ * must match the numDataCache returned from getNumberOfCacheFilesNeeded.
+ * The cache handles will be provided in the same order as with prepareModel_1_2.
* @param token A caching token of length Constant::BYTE_SIZE_OF_CACHE_TOKEN
* identifying the prepared model. It is the same token provided when saving
- * the cache files with IPreparedModel::saveToCache. Tokens should be chosen
+ * the cache files with prepareModel_1_2. Tokens should be chosen
* to have a low rate of collision for a particular application. The driver
* cannot detect a collision; a collision will result in a failed execution
* or in a successful execution that produces incorrect output values.
@@ -263,7 +332,7 @@
* unspecified error
* - INVALID_ARGUMENT if one of the input arguments is invalid
*/
- prepareModelFromCache(handle modelCache, handle dataCache,
+ prepareModelFromCache(vec<handle> modelCache, vec<handle> dataCache,
uint8_t[Constant:BYTE_SIZE_OF_CACHE_TOKEN] token,
IPreparedModelCallback callback)
generates (ErrorStatus status);