About iOS Input Events
What Are Input Events¶
iOS uses the Events concept to handle signals received from different input devices. An Event is an object generated in response to a signal from an input device. These objects are then delivered to the corresponding kernel subsystem, which processes them and notifies all listening processes about taps, key presses, swipes, etc. This means that in order to emulate a signal generated by an external device, such as a touch screen, it is necessary to just send Event objects with the same properties and in the same sequence as they would be generated by a real device.
Simulating a Single Tap¶
The Events API itself is a part of Apple private API, and it is neither open sourced nor documented. The XCTest framework also does not expose any public APIs for input events generation, although there is a possibility to perform events generation via XCTest private undocumented APIs.
In particular, we are interested in the
XCPointerEventPath
and XCSynthesizedEventRecord
interfaces. These APIs allow to create chains of input events and supply them to the system kernel
for execution.
In order to synthesize a single tap, it is necessary to:
- Create a new
XCPointerEventPath
instance and initialize it for touch at the starting point - Add a new
liftUp
event at0.125s
offset usingliftUpAtOffset:
method - Add the generated event path object to
XCSynthesizedEventRecord
instance usingaddPointerEventPath:
method - Execute the events using
synthesizeWithError:
method ofXCSynthesizedEventRecord
instance and control the returned error
There are several limitations to these APIs:
- Each
XCPointerEventPath
instance can only be executed for a single action. If one tries to add, for example, two taps to a single path, then these are effectively ignored - Each
XCPointerEventPath
instance can only be initialized for a particular pointer type: touch, mouse (since Xcode 10.2) or keyboard (since Xcode 10.2) - Events can only be added with increasing offset values to an existing
XCPointerEventPath
instance
More Complicated Actions¶
Unfortunately, because the API is private and has zero documentation, one can only figure out what it can do by playing with it and trying different input combinations.
It is known that providing multiple XCPointerEventPath
instances with overlapping timeouts will
generate a multitouch action with the amount of fingers equal to the amount of the supplied event
paths. So, in order to generate two-finger symmetric swipe we need to supply the following events:
- Create a two
XCPointerEventPath
instances and init them for touch at the starting point - Add a
moveToPoint
event at0.525s
offset usingmoveToPoint:
method to each path - Add a
liftUp
eventa at0.525s
offset usingliftUpAtOffset:
method to each path - Add the generated event paths to
XCSynthesizedEventRecord
instance usingaddPointerEventPath:
method - Execute the events using
synthesizeWithError:
method ofXCSynthesizedEventRecord
instance and control the returned error
Further Reading¶
Unfortunately, there is no information on this topic at all (private API ¯\_(ツ)_/¯
). Consider
visiting the following resources:
- https://github.com/appium/WebDriverAgent/tree/master/PrivateHeaders/XCTest
- https://github.com/appium/WebDriverAgent/blob/master/WebDriverAgentTests/IntegrationTests/FBW3CTouchActionsIntegrationTests.m
- https://github.com/appium/WebDriverAgent/blob/master/WebDriverAgentTests/IntegrationTests/FBW3CMultiTouchActionsIntegrationTests.m
- https://github.com/appium/WebDriverAgent/blob/master/WebDriverAgentLib/Utilities/FBW3CActionsSynthesizer.m