C2SoftAacEnc: Clear encoder internal buffer during flush
During flush call, in some cases, encoders internal buffer
may not be cleared and that leads to an error returned
from the encoder library. This leads to OOB access of
mRemainder buffer.
To fix this, plugin clears encoders internal buffer during
flush.
Also, in case of error during encode call, consume all the
input bytes to prevent OOB access of mRemainder buffer.
bug: 264226775
Test: atest CtsMediaV2TestCases
Test: atest VtsHalMediaC2V1_0TargetAudioEncTest
(cherry picked from https://partner-android-review.googlesource.com/q/commit:7da97785c567f9405811451c8b05da6a9f316646)
Merged-In: I197781c8490e1de2217a2aa019ac780c7873f6d5
Change-Id: I197781c8490e1de2217a2aa019ac780c7873f6d5
diff --git a/media/codec2/components/aac/C2SoftAacEnc.h b/media/codec2/components/aac/C2SoftAacEnc.h
index 9a28280..c79526c 100644
--- a/media/codec2/components/aac/C2SoftAacEnc.h
+++ b/media/codec2/components/aac/C2SoftAacEnc.h
@@ -47,6 +47,9 @@
const std::shared_ptr<C2BlockPool> &pool) override;
private:
+ static constexpr size_t kMaxChannelCount = 6;
+ static constexpr size_t kRemainderBufSize = kMaxChannelCount * sizeof(int16_t);
+
std::shared_ptr<IntfImpl> mIntf;
HANDLE_AACENCODER mAACEncoder;
@@ -63,7 +66,7 @@
std::atomic_uint64_t mOutIndex;
// We support max 6 channels
- uint8_t mRemainder[6 * sizeof(int16_t)];
+ uint8_t mRemainder[kRemainderBufSize];
size_t mRemainderLen;
status_t initEncoder();