Allow multi-window multiple device streams
Before this CL, only one device could be active in the system at a given
time. This was enforced early inside the dispatcher code.
In this CL, this restriction is removed, and a new restriction is
introduced:
- At most one input device can be active for a given connection
That means that it's now possible to touch one window and draw with
stylus in another.
This is implemented by moving the "changed device" check from the
TouchState into InputState.
This should work OK because there will be a unique ViewRootImpl per
connection, and the UI toolkit will continue to process the events in a
single-device mode.
In the future, we can consider enabling same-window multi-device
streams. This would likely require a new public API.
After this CL, the dispatcher will work in the following manner:
TouchState -> always works in the multi-device mode. Assumes 1 window
can receive multiple pointers from different devices.
InputState (per-connection) -> acts as a rejector / multiplexor. It only
selects a single device to be active for the given connection. This is
done so that in the future, we can potentially turn off the "single
device active" behaviour of InputState without having to change other
parts of the dispatcher.
What happens if there are multiple device streams being sent?
- In general, latest device always wins
- Exception: stylus always takes precedence over other devices
- Latest stylus device cancels the current stylus device
One other behaviour there:
- If for the same device id, source changed (this is one of the tests),
then the current gesture is canceled.
Additional changes:
The case of touch pointer down -> mouse down -> second touch pointer
down does not cancel mouse. For simplicity, we just wait for a new touch
gesture to start before canceling the current mouse gesture.
Pilfer pointers changes:
Previously, two devices being active in one window cause pilferPointers
to fail. The new behaviour is that all of the active gestures in the spy
window that's doing the pilfering are going to get pilfered.
So if a spy window is receiving mouse and touch, then both mouse and
touch gestures are going to be pilfered (because the windows below the
spy would be getting independent mouse and touch streams). In practice,
in this CL the spy will never receive two devices at the same time,
because we are only allowing single device to be active per-window. But
the understanding here is that eventually, we may want to have multiple
devices going to the spy.
How same-token windows are handled:
During TouchState computation, the windows with the same token are
treated separately (since they do have a different layer id). Then
later, during TouchState -> InputTarget conversion, they are all lumped
into one target. One problem with this approach is that sometimes, we
want to generate HOVER_EXIT with response to receiving ACTION_DOWN
event. This is because InputReader today doesn't always generate
HOVER_EXIT prior to sending ACTION_DOWN. That means that the dispatcher
has to have a special logic for dealing with these cases. The approach
taken in this CL is to force-generate new DISPATCH_AS_HOVER_EXIT input
targets, which would allow them to remain separate from DISPATCH_AS_IS
input targets for the ACTION_DOWN event.
This avoid the collision of pointers. Otherwise, we would add a pointer
id for the HOVER_EXIT event, which would not be valid for the
ACTION_DOWN event's pointer id.
Bug: 211379801
Test: TEST=inputflinger_tests; m $TEST && $ANDROID_HOST_OUT/nativetest64/$TEST/$TEST
Change-Id: If2eae87bc2a40b61144ddcd019a9800c2d526072
diff --git a/services/inputflinger/dispatcher/TouchState.cpp b/services/inputflinger/dispatcher/TouchState.cpp
index 9dcf615..2ead171 100644
--- a/services/inputflinger/dispatcher/TouchState.cpp
+++ b/services/inputflinger/dispatcher/TouchState.cpp
@@ -14,6 +14,7 @@
* limitations under the License.
*/
+#include <android-base/logging.h>
#include <android-base/stringprintf.h>
#include <gui/WindowInfo.h>
@@ -31,15 +32,6 @@
*this = TouchState();
}
-std::set<int32_t> TouchState::getActiveDeviceIds() const {
- std::set<int32_t> out;
- for (const TouchedWindow& w : windows) {
- std::set<int32_t> deviceIds = w.getActiveDeviceIds();
- out.insert(deviceIds.begin(), deviceIds.end());
- }
- return out;
-}
-
bool TouchState::hasTouchingPointers(DeviceId deviceId) const {
return std::any_of(windows.begin(), windows.end(), [&](const TouchedWindow& window) {
return window.hasTouchingPointers(deviceId);
@@ -65,9 +57,9 @@
}
}
-void TouchState::clearHoveringPointers() {
+void TouchState::clearHoveringPointers(DeviceId deviceId) {
for (TouchedWindow& touchedWindow : windows) {
- touchedWindow.clearHoveringPointers();
+ touchedWindow.removeAllHoveringPointersForDevice(deviceId);
}
clearWindowsWithoutPointers();
}
@@ -82,9 +74,16 @@
ftl::Flags<InputTarget::Flags> targetFlags, DeviceId deviceId,
std::bitset<MAX_POINTER_ID + 1> touchingPointerIds,
std::optional<nsecs_t> firstDownTimeInTarget) {
+ if (touchingPointerIds.none()) {
+ LOG(FATAL) << __func__ << "No pointers specified for " << windowHandle->getName();
+ return;
+ }
for (TouchedWindow& touchedWindow : windows) {
// We do not compare windows by token here because two windows that share the same token
- // may have a different transform
+ // may have a different transform. They will be combined later when we create InputTargets.
+ // At that point, per-pointer window transform will be considered.
+ // An alternative design choice here would have been to compare here by token, but to
+ // store per-pointer transform.
if (touchedWindow.windowHandle == windowHandle) {
touchedWindow.targetFlags |= targetFlags;
if (targetFlags.test(InputTarget::Flags::DISPATCH_AS_SLIPPERY_EXIT)) {
@@ -237,14 +236,16 @@
return *it;
}
-bool TouchState::isDown() const {
- return std::any_of(windows.begin(), windows.end(),
- [](const TouchedWindow& window) { return window.hasTouchingPointers(); });
+bool TouchState::isDown(DeviceId deviceId) const {
+ return std::any_of(windows.begin(), windows.end(), [&deviceId](const TouchedWindow& window) {
+ return window.hasTouchingPointers(deviceId);
+ });
}
-bool TouchState::hasHoveringPointers() const {
- return std::any_of(windows.begin(), windows.end(),
- [](const TouchedWindow& window) { return window.hasHoveringPointers(); });
+bool TouchState::hasHoveringPointers(DeviceId deviceId) const {
+ return std::any_of(windows.begin(), windows.end(), [&deviceId](const TouchedWindow& window) {
+ return window.hasHoveringPointers(deviceId);
+ });
}
std::set<sp<WindowInfoHandle>> TouchState::getWindowsWithHoveringPointer(DeviceId deviceId,