Fix the order of applying layer alpha and color matrices in GLESRenderEngine
If the system want to do a color inversion,
there is an unexpected result due to applying the layer alpha first.
Solution: RenderEngine should apply
the layer's alpha after multiplying by the color matrix.
For example, given the following color inversion matrix:
| r.r r.g r.b | | -1.0 0.0 0.0 |
| g.r g.g g.b | = | 0.0 -1.0 0.0 |
| b.r b.g b.b | | 0.0 0.0 -1.0 |
| Tr Tg Tb | | 1.0 1.0 1.0 |
We want to get a black layer when applying the color transform to a white layer;
However, the white layer applies layer alpha before color matrix. We can get
the below calculation: When layer alpha is 0, R_out value was always Tr.
Then we do not get a black layer when the system applies color inversion to a
white layer.
R_out = (R_in * layer_alpha) * r.r + Tr
Therefore we apply the color matrix first, so we can get the below calculation.
When the layer alpha is 0, we can get a black layer.
R_out = (R_in * r.r + Tr) * layer_alpha
= (R_in * r.r * layer_alpha) + (Tr * layer_alpha)
adds RenderEngineTest::fillBufferColorTransformZeroLayerAlpha
Bug: 157204341
Test: color inversion when switching pages in the Setting app
Test: RenderEngineTest
Change-Id: I206184aae84042290b00e411709e04e54579780a
diff --git a/libs/renderengine/gl/ProgramCache.cpp b/libs/renderengine/gl/ProgramCache.cpp
index 7fc0499..5ff9240 100644
--- a/libs/renderengine/gl/ProgramCache.cpp
+++ b/libs/renderengine/gl/ProgramCache.cpp
@@ -740,15 +740,6 @@
if (needs.isOpaque()) {
fs << "gl_FragColor.a = 1.0;";
}
- if (needs.hasAlpha()) {
- // modulate the current alpha value with alpha set
- if (needs.isPremultiplied()) {
- // ... and the color too if we're premultiplied
- fs << "gl_FragColor *= color.a;";
- } else {
- fs << "gl_FragColor.a *= color.a;";
- }
- }
}
if (needs.hasTransformMatrix() || (needs.getInputTF() != needs.getOutputTF()) ||
@@ -768,6 +759,23 @@
}
}
+ /*
+ * Whether applying layer alpha before or after color transform doesn't matter,
+ * as long as we can undo premultiplication. But we cannot un-premultiply
+ * for color transform if the layer alpha = 0, e.g. 0 / (0 + 0.0019) = 0.
+ */
+ if (!needs.drawShadows()) {
+ if (needs.hasAlpha()) {
+ // modulate the current alpha value with alpha set
+ if (needs.isPremultiplied()) {
+ // ... and the color too if we're premultiplied
+ fs << "gl_FragColor *= color.a;";
+ } else {
+ fs << "gl_FragColor.a *= color.a;";
+ }
+ }
+ }
+
if (needs.hasRoundedCorners()) {
if (needs.isPremultiplied()) {
fs << "gl_FragColor *= vec4(applyCornerRadius(outCropCoords));";