[Rounded Corners] Take display rotation into account.
Previously when we applied glScissor on the layer, we always assume after the
transformation, the layer already had the aligned coordinates. However, we
still need to take rotation into account such that the (left, top) and (right,
bottom) coordinates align with the display coordination space.
BUG: 129619488
Test: Build, flash and boot. Verify with different rotation.
Change-Id: I6db6d5465af3e1961477960d9cba5c7331fa1ba1
diff --git a/libs/renderengine/gl/GLESRenderEngine.cpp b/libs/renderengine/gl/GLESRenderEngine.cpp
index d0127b8..7c55133 100644
--- a/libs/renderengine/gl/GLESRenderEngine.cpp
+++ b/libs/renderengine/gl/GLESRenderEngine.cpp
@@ -743,6 +743,9 @@
// We separate the layer into 3 parts essentially, such that we only turn on blending for the
// top rectangle and the bottom rectangle, and turn off blending for the middle rectangle.
FloatRect bounds = layer.geometry.roundedCornersCrop;
+
+ // Firstly, we need to convert the coordination from layer native coordination space to
+ // device coordination space.
const auto transformMatrix = display.globalTransform * layer.geometry.positionTransform;
const vec4 leftTopCoordinate(bounds.left, bounds.top, 1.0, 1.0);
const vec4 rightBottomCoordinate(bounds.right, bounds.bottom, 1.0, 1.0);
@@ -750,8 +753,28 @@
const vec4 rightBottomCoordinateInBuffer = transformMatrix * rightBottomCoordinate;
bounds = FloatRect(leftTopCoordinateInBuffer[0], leftTopCoordinateInBuffer[1],
rightBottomCoordinateInBuffer[0], rightBottomCoordinateInBuffer[1]);
- const int32_t radius = ceil(layer.geometry.roundedCornersRadius);
+ // Secondly, if the display is rotated, we need to undo the rotation on coordination and
+ // align the (left, top) and (right, bottom) coordination with the device coordination
+ // space.
+ switch (display.orientation) {
+ case ui::Transform::ROT_90:
+ std::swap(bounds.left, bounds.right);
+ break;
+ case ui::Transform::ROT_180:
+ std::swap(bounds.left, bounds.right);
+ std::swap(bounds.top, bounds.bottom);
+ break;
+ case ui::Transform::ROT_270:
+ std::swap(bounds.top, bounds.bottom);
+ break;
+ default:
+ break;
+ }
+
+ // Finally, we cut the layer into 3 parts, with top and bottom parts having rounded corners
+ // and the middle part without rounded corners.
+ const int32_t radius = ceil(layer.geometry.roundedCornersRadius);
const Rect topRect(bounds.left, bounds.top, bounds.right, bounds.top + radius);
setScissor(topRect);
drawMesh(mesh);