NNAPI Burst -- HAL interface
FastMessageQueue is a Treble-compliant data structure that enables fast
communication between two processes. The FMQ object itself is an atomic
circular buffer that is optionally synchronized with a futex. However,
FMQ has no notion of ownership or lifetime across processes, so it must
be paired with higher-level constructs to manage the lifetime and
ownership.
The NNAPI is introducing the notion of an "Execution Burst" object (or
more simply a "Burst" object), which is similar to an
ANeuralNetworksExecution, but is intended to be reused across multiple
executions and has lower IPC overheads. It achieves this low IPC
overhead by replacing HIDL HwBinder calls with FMQ messages.
Specifically, it replaces IPreparedModel::executeSynchronously's call
from the client into the service with fmq_sync<FmqRequestDatum> (an FMQ
channel used to pass a serialized Request object) and it replaces
the return from the service into the client with
fmq_sync<FmqResultDatum> (an FMQ channel used to return serialized
result status and OutputShapes information).
Each channel is a unidirectional flow of information with exactly one
producer and exactly one consumer. The channels are created by the NN
runtime and passed to the service via
IPreparedModel::configureExecutionBurst.
This CL defines the FmqRequestDatum and FmqResultDatum types in
types.hal. IBurstContext.hal defines IBurstContext, a HIDL object used
by the service to manage the resources of a Burst. IBurstCallback.hal
defines IBurstCallback, a HIDL callback object that can be used to
retrieve the handle to a resource the service has either not yet seen or
has evicted from its cache. Finally, IPreparedModel.hal is extended with
IPreparedModel::configureExecutionBurst to create the burst object.
Bug: 119570067
Test: mma
Change-Id: I333da70201531b1396efc714d096c277e8e1d47b
Merged-In: I333da70201531b1396efc714d096c277e8e1d47b
(cherry picked from commit 7e91e24fe155de15c677c48ca5a2c141ba2246dc)
diff --git a/neuralnetworks/1.2/IPreparedModel.hal b/neuralnetworks/1.2/IPreparedModel.hal
index 4e91c67..2d4e572 100644
--- a/neuralnetworks/1.2/IPreparedModel.hal
+++ b/neuralnetworks/1.2/IPreparedModel.hal
@@ -19,6 +19,8 @@
import @1.0::ErrorStatus;
import @1.0::IPreparedModel;
import @1.0::Request;
+import IBurstCallback;
+import IBurstContext;
import IExecutionCallback;
/**
@@ -100,11 +102,47 @@
* - NONE if execution is performed successfully
* - DEVICE_UNAVAILABLE if driver is offline or busy
* - GENERAL_FAILURE if there is an unspecified error
- * - OUTPUT_INSUFFICIENT_SIZE if provided output buffer is
- * not large enough to store the resultant values
+ * - OUTPUT_INSUFFICIENT_SIZE if at least one output
+ * operand buffer is not large enough to store the
+ * corresponding output
* - INVALID_ARGUMENT if one of the input arguments is
* invalid
+ * @return outputShapes A list of shape information of model output operands.
+ * The index into "outputShapes" corresponds to the index
+ * of the output operand in the Request outputs vector.
+ * outputShapes must be empty unless the status is either
+ * NONE or OUTPUT_INSUFFICIENT_SIZE.
*/
executeSynchronously(Request request)
- generates (ErrorStatus status);
+ generates (ErrorStatus status, vec<OutputShape> outputShapes);
+
+ /**
+ * Configure a Burst object used to execute multiple inferences on a
+ * prepared model in rapid succession.
+ *
+ * @param callback A callback object used to retrieve memory resources
+ * corresponding to a unique identifiers ("slots").
+ * @param requestChannel Used by the client to send a serialized Request to
+ * the Burst for execution. requestChannel must not be
+ * used to pass a second Request object until a result
+ * has been received from resultChannel.
+ * @param resultChannel Used by the service to return the results of an
+ * execution to the client: the status of the execution
+ * and OutputShape of all output tensors. resultChannel
+ * must be used to return the results if a Request was
+ * sent through the requestChannel.
+ * @return status Error status of configuring the execution burst, must be:
+ * - NONE if the burst is successfully configured
+ * - DEVICE_UNAVAILABLE if driver is offline or busy
+ * - GENERAL_FAILURE if there is an unspecified error
+ * - INVALID_ARGUMENT if one of the input arguments is
+ * invalid
+ * @return context Object containing all resources (such as cached
+ * hidl_memory) related to a Burst if successful, otherwise
+ * nullptr.
+ */
+ configureExecutionBurst(IBurstCallback callback,
+ fmq_sync<FmqRequestDatum> requestChannel,
+ fmq_sync<FmqResultDatum> resultChannel)
+ generates (ErrorStatus status, IBurstContext context);
};