skip to content

Touch and Motion

Three ways a movement across the screen can be expressed to a driver: a portable pointer sequence, a driver's own named gesture command, or a scroll that hunts for an off-screen element.

on this pageshow

explore

questions

13

In Appium, what does a POST /session/:sessionId/actions body contain to tap one point?

level: juniorimportance: must knowfreq 74%

answer

  1. one array of input sources
  2. the source says who is touching
  3. touch, not mouse, on device
  4. move, then down, then up

basics

~20 s

One pointer input source, and inside it the ticks that model the finger. The source declares an id and a pointer type of touch; the ticks are a move to the coordinate, a press and a lift.

solid answer

~40 s

The body is a single `actions` array of **input sources**. A tap needs one entry with `type` of `"pointer"`, an arbitrary `id` such as `finger1`, and `parameters` of `{"pointerType": "touch"}` — touch is the input both Android's UiAutomator2 driver and Apple's XCUITest driver inject on a native session. That source carries its own nested `actions` list of **ticks**: a `pointerMove` giving `x`, `y` and an `origin` (the `viewport`, the current `pointer` position, or an element id, which lets the driver hit the element's centre for you), then `pointerDown` with `button` 0, then `pointerUp` with `button` 0. The whole thing goes in one request — there is no round trip per tick.

code

json · 14 lines
json
{
  "actions": [
    {
      "type": "pointer",
      "id": "finger1",
      "parameters": { "pointerType": "touch" },
      "actions": [
        { "type": "pointerMove", "duration": 0, "origin": "viewport", "x": 540, "y": 640 },
        { "type": "pointerDown", "button": 0 },
        { "type": "pointerUp", "button": 0 }
      ]
    }
  ]
}

go deeper

for a junior

Be able to name the three ticks a tap needs — a move, a press and a lift — and say that they travel together in one request to the actions endpoint.

for a middle

Explain the two nesting levels: an array of input sources, each with its own array of ticks, and why the pointer type is touch on a native mobile session.

for a senior

Show how you keep chains device-independent by preferring element origins over pixel coordinates, and say when a hand-built chain is worth its maintenance cost at all.

for a principal

Own the call on whether the suite standardises on one portable pointer vocabulary across Android and Apple devices or accepts two per-driver branches, and what each choice costs over time.

## What the request looks like Appium sends every hand-built gesture to one endpoint: `POST /session/:sessionId/actions`. The body is an object with a single `actions` key whose value is an array of **input sources** — the virtual devices doing the input. A tap needs exactly one source, and that source is where the payload's real content lives. A source has three parts. `type` says what kind of device it is; for a touch gesture that is `"pointer"`. `id` is an arbitrary name you choose, such as `finger1`; it exists so the driver can tell sources apart when a gesture uses more than one. `parameters` carries `{"pointerType": "touch"}` — the pointer is a finger, not a mouse and not a stylus. On a native session driven by Android's UiAutomator2 driver or by Apple's XCUITest driver, touch is the input the device actually receives, so declare it explicitly rather than leaving the device kind to be inferred. ## The ticks: what the finger does Nested inside the source is a second `actions` array. These are the **ticks** — the ordered instants of the gesture. A tap is three of them: 1. `pointerMove`, carrying `x`, `y`, an `origin` and a `duration`. This is how the finger reaches the target; the duration is travel time, and zero is normal for a tap. 2. `pointerDown`, carrying `button` 0. Contact begins. 3. `pointerUp`, carrying `button` 0. Contact ends. All three travel in the same request. There is no round trip per tick, and no way to hold the pointer across two separate calls: the sequence is dispatched and the pointer is up again by the time the response comes back. A dwell, if you want one, is a `pause` tick placed inside that same list. ## Choosing an origin `origin` decides what `x` and `y` are measured against, and it is the field that decides whether your gesture survives a second device: - `viewport` — absolute coordinates on screen. Simple, and brittle: a scaffold-register row that sits at y 640 on one handset is somewhere else on the next. - `pointer` — coordinates relative to where the pointer already is, which is how you express "move 300 pixels up from here" inside a swipe. - an **element id** returned by a find — the driver resolves that element's centre for you. This is the form to prefer for a tap in the scaffolding-inspection app: locate the scaffold row, then move to it, and the payload stops encoding one device's geometry. ## One payload, two engines The payload is identical on both platforms; what differs is who executes it. | | Android's UiAutomator2 driver | Apple's XCUITest driver | |---|---|---| | what receives the sequence | a helper server the driver pushes to the device and starts | WebDriverAgent, which the driver builds, installs and launches | | the request you send | `POST /session/:sessionId/actions` | `POST /session/:sessionId/actions` | | the pointer source | one `pointer` source, `pointerType` of `touch` | one `pointer` source, `pointerType` of `touch` | That sameness is the whole point of the endpoint. Each driver also has named gesture commands of its own, but those diverge by name and by signature between the two platforms; the pointer sequence is the vocabulary you can write once and send to either. ## The mistakes that cost the most time - **Splitting the ticks across requests.** Three calls give you a move, a tap and a stray release — not one tap. - **Hard-coding viewport coordinates** because the first device passed, then discovering the suite only works on that device. - **Omitting `button` on `pointerDown` and `pointerUp`** and assuming something sensible will be supplied. - **Assuming the response proves the tap landed.** It reports that the sequence was dispatched; whether the scaffold row opened is something your assertions have to check. - **Reaching for a chain when a plain element click would do.** The standard element click command is simpler and more readable; the actions endpoint earns its keep for gestures a click cannot express. ## How to read a payload you did not write Work outside in. The outer `actions` array tells you how many fingers are involved. Each source's `parameters` tells you what kind of input device it is. Only then read the inner `actions` list, left to right, as a story in time: where the finger went, when it pressed, what it did while it was down, and when it lifted. Almost every gesture bug in a hand-written chain is visible at that third level, and almost none of them are visible at the first.

  • What does setting origin to an element id buy you over viewport coordinates?
    The driver resolves the element's centre itself, so the chain survives a different screen size, a layout shift or a scrolled list. Viewport coordinates encode one device's geometry into the test; an element origin encodes the intent instead, which is what you want a tap on a scaffold row to mean.
  • Why is this payload the portable one across Android and Apple devices?
    Because both drivers accept the same pointer source and inject it as touch input, so one chain runs on either platform. The named gesture commands each driver ships are per-driver: they differ in name and in signature, so using them means writing and maintaining two branches.

saying these in an interview costs you the question

  • Sends one HTTP request per tick in the sequence
  • Leaves the pointer type unset and assumes touch
  • Thinks x and y are always absolute screen coordinates
  • Believes an element id cannot be used as an origin
  • Treats a successful response as proof the tap landed
open as a page

In Appium, which command scrolls an off-screen row into view on Android, and which on iOS?

level: juniorimportance: must knowfreq 72%

basics

~10 s

On Android the UiAutomator2 driver scrolls while it searches: an -android uiautomator selector running UiScrollable scrollIntoView, or mobile: scroll. On iOS, XCUITest scrolls to an element you have already found, with mobile: scrollToElement.

open as a page

In Appium 3, what replaced the touch/perform endpoints, and is TouchAction gone?

level: middleimportance: must knowfreq 66%

basics

~20 s

The W3C Actions API replaced them: gestures now go to the actions endpoint. TouchAction is not gone from the clients — java-client still ships it marked deprecated — but the Appium 3 server no longer routes the old touch paths.

open as a page

In Appium, why does a pointerDown then pointerUp action chain tap instead of long-pressing?

level: seniorimportance: must knowfreq 58%

basics

~20 s

Nothing in the chain holds the finger down. A pointer sequence advances one tick at a time, so a press is followed immediately by the lift. Insert a pause tick carrying the dwell between them.

open as a page

How would you build one Appium scroll-into-view step for a pet-grooming app on Android and iOS?

level: seniorimportance: must knowfreq 46%

basics

~10 s

Share the shape, not the command. One helper takes a logical target and returns something reachable, branching inside: Android builds a UiScrollable scrollIntoView selector, iOS finds the row and then posts mobile: scrollToElement.

open as a page

In Appium on iOS, which attribute must a row report before XCUITest will tap it, and how do you get there?

level: seniorimportance: must knowfreq 58%

basics

~20 s

Hittable. XCUITest acts on an iOS element only when a hit test at its activation point resolves to that element. mobile: scrollToElement scrolls its container until it is hittable; an overlay such as a sticky bar must be cleared instead.

open as a page

In Appium, which driver ships mobile: swipeGesture and which ships mobile: swipe?

level: juniorimportance: should knowfreq 55%

basics

~10 s

Appium's UiAutomator2 driver on Android ships mobile: swipeGesture; the XCUITest driver on Apple platforms ships mobile: swipe. The names are not shared, so a helper written for one driver fails outright on the other.

open as a page

In Appium, what arguments do UiAutomator2's mobile: swipeGesture and XCUITest's mobile: swipe take?

level: middleimportance: should knowfreq 44%

basics

~20 s

UiAutomator2's mobile: swipeGesture needs a target element or rectangle, a direction and a mandatory percent, plus an optional speed. XCUITest's mobile: swipe takes an optional element, a direction and an optional velocity, and has no percent at all.

open as a page

Why does Appium's UiScrollable scrollIntoView find an Android row that findElement cannot?

level: middleimportance: should knowfreq 52%

basics

~20 s

UiScrollable runs on the device: the UiAutomator2 server swipes the Android list and rescans after each swipe. A plain findElement is one query against the hierarchy as it exists now, and a recycled off-screen row is not in it.

open as a page

How would you design one Appium gesture layer when UiAutomator2 and XCUITest share no command names?

level: principalimportance: should knowfreq 29%

basics

~20 s

Put the per-driver mapping in one adapter behind a small facade, keep only genuinely corresponding gestures in the portable vocabulary, and make an unsupported gesture fail loudly rather than be approximated. Log the command actually posted.

open as a page

In Appium, what does DELETE /session/:sessionId/actions do after a mobile gesture?

level: seniorimportance: nice to knowfreq 27%

basics

~10 s

Effectively nothing on Appium's mobile drivers. The command releases input state a session is still holding, and these drivers keep none between requests, so it is not a cleanup hook you can build on.

open as a page

Your Appium mountain-hut suite posts mobile: swipeGesture to Android and iOS sessions alike — what breaks?

level: seniorimportance: nice to knowfreq 36%

basics

~20 s

The iOS session runs the XCUITest driver, which declares no mobile: swipeGesture, so that call fails on every run. Gesture commands are per driver: branch the helper on the driver in play and post mobile: swipe on the Apple side.

open as a page

In Appium, where does a scroll-into-view step fail for a missing row on Android versus iOS?

level: seniorimportance: nice to knowfreq 34%

basics

~20 s

Android fails late: the UiAutomator2 server spends its whole search-swipe budget before the find reports nothing. iOS fails early, because mobile: scrollToElement needs an element handle, so the find fails first and the scroll never runs at all.

open as a page