Apple’s new UITest suite has some developers excited, and others disappointed in lost functionality. UITest works differently than the functional testing solutions developers have come to rely on, such as KIF. Instead of giving you access to elements themselves, UITest gives access to proxy elements with minimal parameters to interact with. Learning to separate unit testing and unit based uitesting from functional testing can be a frustrating experience, especially when working with a new framework that still has its own kinks to work out. It is my hope the following guide will help developers form some clarity into the methodology needed when working with UITest.

 * This article was written in mid December 2015. At that point UITest was still experiencing several bugs. Many times I had tests fail that the next day would pass after no changes. It is recommend you clean and delete derived data often, as well as restart XCode from time to time. You may also have to find some creative workarounds. More in the "Gotchas" section below. When looking at the docset keep in mind that UITest is for both iOS and OSX testing and therefore there may be OSX specific items mentioned (click vs tap for example).

 

How It Works

UITest is unique in that it does not use the project code to test, it exists outside the app. UITest instead looks at what is available in the simulator and returns to us instances of XCUIElement based on what it finds, XCUIElement and XCUIApplication are the proxies that are used for this. A button is a XCUIElement of type XCUIElementTypeButton (or just .Button), for example. This means you are not able to access ObjC or Swift classes/objects nor their associated properties. You can only access XCUIElement properties. This also means there is no @testable or other such imports as in XCTest.

UITest includes a new Record feature which can be used to record actions and help speed up the writing of tests. UITests can be written without the help of the Record feature, and therefore can be part of test-driven development, however when writing tests after writing the code you are testing it is best to first use Record and then modify the results. This is because it is sometimes difficult to predict how an element will be exposed to UITest until you are more familiar with it. It also helps to identify issues you may have been unaware of such as multiple items with the same identifier.

XCUIApplication is the proxy that is used for testing. It launches a new instance of the app for every test giving you a clean slate to run your test on. As with XCTest there are setup and teardown methods and you can encapsulate common functionality such as writing a function in the test file to clear a text field which you can then call in each subsequent test that is needed. You can also make an extension on XCUIElement to reuse code in any part of the test suite.

XCUIApplication launches and terminates the app (launch and terminate, respectively). You can also pass arguments to the application on launch using launchArguments and pass an environment using launchEnvironment. From the header files / docset: "Unlike NSTask, it is legal to modify the environment [ and launch arguments ] after the application has been launched. These changes will not affect the current launch session, but will take effect the next time the application is launched."

UITest is built on top of the XCTest Framework and includes new additions to the API:

 

 

There aren't any official docs but Joe Masilotti has compiled docs based on the framework headers. The items above link directly to those docs. If the author takes the website down we have a copy of the docset forked on Github (as of Dec 8, 2015). The docset also includes the rest of the XCTest framework, not only UITest.

 

Identifying Elements -> XCUIElementQuery

Elements are identified using queries, most often using a combination of identifiers and types. Elements are unique, in order to use an element the query must return one element. Apple has provided several shortcut queries (app.tables returns a XCUIElementQuery of all the tables and app.table["table identifier"] would return the XCUIElement of type .Table with the specified identifier vs the long form which would be app.containingType(.Table, identifier: "table identifier")) and one can use the long form or the short form depending on preference and need. Queries are chained together and variable names can be substituted throughout the chain:

let app = XCUIApplication()
let cellQuery = app.tables.cells.containingType(.StaticText, identifier:"String Identifier")
let helpButton = cellQuery.buttons["?"]
helpButton.tap()

There are often multiple ways to reach an element, for instance a cell with identifier "Purchase Cell" that contains a button "Buy Item" and also a label with identifier "Purchase" could be found several ways and if you use Record you should be presented with a clickable token that allows you to choose which version you would like to use:

let cell = XCUIApplication().tables.cells["Purchase Cell"]
let cell = XCUIApplication().tables.cells.containingType(.Button, identifier: "Buy Item")
let cell = XCUIApplication().tables.cells.containingType(.StaticText, identifier: "Purchase")
let cell = XCUIApplication().descendantsMatchingType(.Table).element.descendantsMatchingType(.Cell).matchingIdentifier("Purchase Cell")

If you were wanting to interact with the label or button rather than the cell you could do the following:

let buyButton = cell.buttons["Buy Item"]
let purchaseLabel = cell.staticTexts["Purchase"]

These are short forms, the long form would be:

let buyButton = cell.childrenMatchingType(.Button, identifier: "Buy Item")

Or if you did not have a way to identify the cell but you could identify the table you can also do this:

let buyButton = table.descendantsMatchingType(.Button, identifier: "Buy Item")

If you had a cell that had two buttons with the same identifier you would need to identify which button you would like by it's index value. Using Record is recommended as in practice I found the elements were not always at the expected indices. Note "elementAtIndex" will autocomplete but is already deprecated, use elementBoundByIndex instead.

let button = XCUIApplication().tables.cells["Purchase Cell"].buttons.elementBoundByIndex(1)

This would find the button at index 1 out of all buttons on the cell.

let button = XCUIApplication().tables.cells["Purchase Cell"].buttons["Buy Item"].elementBoundByIndex(1)

This would find the button at index 1 out of all buttons on the cell with identifier "Buy Item".

If you have only one item of a certain type there is an additional shortcut, element. For example if you have a cell containing one button, that button can be accessed like so:

let button = cellQuery.buttons.element

For element identification there is additionally elementMatchingPredicate(NSPredicate) and elementMatchingType(XCUIElementType, identifier: String?). These are separate from

containingPredicate(NSPredicate) and containingType(XCUIElementType, identifier: String?) which are checking the element for items inside it whereas the elementMatching... options are checking the values on the element itself. Finding the correct element will often include combinations of several query attributes.

If you have difficulty interacting with an element, printing the accessibility hierarchy can help (print(app.debugDescription), and if that doesn't work here is a tip from shinobicontrols.com:


 Sometimes when you tap on an element while recording, you’ll notice that the code produced doesn’t look quite right. This is usually because the element you are interacting with is not visible to Accessibility. To find out if this is the case, you can use XCode’s Accessibility Inspector.

accessibilityInspector.png

Once it is open, if you hit CMD+F7 and hover over an element with your mouse in the simulator, then you’ll see comprehensive information about the element underneath the cursor. This should give you a clue about why Accessibility can’t find your element.

https://www.shinobicontrols.com/blog/ios9-day-by-day-day2-ui-testing


 

Queries Reference:

Query
For
Elements
Directly
*returns a query
objectForKeyedSubscript: / object["subscript'] = matches identifier only
matchingIdentifier:
matchingType:identifier:
matchingPredicate:
containingType:identifier:
containingPredicate:
Query
Elements
in the
Tree
*returns a query
descendantsMatchingType:
childrenMatchingType:
Access
Elements
Directly
element = returns an element
elementBoundByIndex: = returns an element
elementMatchingPredicate: = returns an element
elementMatchingType:identifier: = returns an element
allElementsBoundByAccessibilityElement = returns an array
allElementsBoundByIndex = returns an array
Available
Query
Actions
count = returns number of matches found for the query
debugDescription = can be used on an element or a query

 

Sharing Code Between App and UITest Targets

UITest is a completely separate entity from the app itself and therefore can only access UI elements however it may be beneficial to share code between the project and UITest target for writing your tests. To use code from the app target there are two options.

  1.  Add the file to Compile Sources (or check off UITest target in member dependencies on the file). Must remember to also compile all files used by this file whether or not the test will use them.
  2.  Take the shared code out of the app's files and into a new file and share that file between the two. This eliminates the need to also compile dependent files.

A good example of when this would be useful is given by Big Nerd Ranch where they show how you can use shared code to check if a to-do item is marked as finished (strikethrough). While we can not access the object's properties directly we can use shared code to return values that we can use for assertions.

 

Properties, Attributes, Methods -> XCUIElement

UITest uses the accessibility API to populate attributes for the XCUIElement under test. From Masilotti's docs:

Attributes Reference:

Properties identifier = the accessibility identifier
frame = the frame of the element in the screen coordinate space
value = the raw value attribute of the element, type varies
title = the title attribute of the element
label = the label attribute of the element
elementType = from XCUIElementType
enabled = whether or not the element is enabled for user interaction
horizontalSizeClass = the horizontal size class of the element
verticalSizeClass = the vertical size class of the element
placeholderValue = the value that is displayed when the element has no value
selected = whether or not the element is selected
hasFocus = whether or not the element has UI focus
Methods exists = whether or not the element exists (use this; XCTAssertNotNil will always pass)
hittable = whether or not a hit point can be computed for the element
coordinateWithNormalizedOffset: = computes screen point with offset
 
 
 

Events Reference:

Gestures tap
doubleTap
twoFingerTap
tapWithNumberOfTaps:numberOfTouches:
pressForDuration:
pressForDuration:thenDragToElement:
swipeUp
swipeDown
swipeLeft
swipeRight
pinchWithScale:velocity: = scale of 0-1 pinch close, >1 is zoom out, and velocity is in factor per second
rotate:withVelocity: = rotation in radians, velocity in radians per second
scrollByDeltaX:deltaY: = scroll the view the specified pixels, x and y.
Keyboard Can also use .tap() on keyboard keys
typeText:
typeKey:modifierFlags: = types a single key with the specified modifier flags.
Slider adjustToNormalizedSliderPosition: = adjustment is a "best effort", range is 0-1
normalizedSliderPosition = returns position, range is 0-1
*slider.value will return the value instead of the position
Picker adjustToPickerWheelValue:

 

How-To's and Gotchas

In the resource links below Joe Masilotti has several code examples including reordering cells, pull to refresh, check if an element is on screen, and handling system alerts.

The following are things myself and others from Metova have come across:

 

Frames For Off-Screen Elements

Queries will return elements which are offscreen. If the frame is needed for test logic or an assertion, before getting the frame make sure it is on screen, usually by tapping on it first.

 

Asserting Adjusted Slider Values

adjustToNormalizedSliderPosition does not guarantee accuracy but rather is an estimate. I found setting 0.0 or 1.0 does give 0% and 100% respectively but otherwise it is best to XCTAssertNotEqual to the previous value rather than test the new value.

 

Inconsistent Test Failures

UITest is still buggy. Scrolling to a cell does not work consistently. In my tests I had it fail 20-30% of the time one day and then on another day not at all. I witnessed the same thing happen to another developer as well. I also experienced tests failing one day and passing the next, with no changes to the code. If you can not figure out why a test is failing I recommend moving onto another task and returning to debug at the end. When you come back you may discover it is a passing test after all.

 

Record Returns Wrong Element

UITest will sometimes grab an image instead of a button when tapping on a button. I also noticed that UITest prefers to return Static Texts (labels) rather than buttons, this may have to do with the view hierarchy and what element is on top.

 

Slider Will Not Move

Slider only slides using adjustToNormalizedSliderPosition if it starts at 0. A work around is to use pressForDuration:thenDragToElement and find an element nearby. Remember this may not be accurate unless you are dragging to an element at the extreme beginning (0%) or end (100%)

 

Multiple Matches Failure with Recorded Code

Sometimes Record does not identify elements correctly and a test will fail due to multiple matches or no matches when Record gives you code indicating there is one unique match. I found the multiple matches error happened most often when there were multiple matches to the query but the other elements were off screen at the time of Recording. The best way around this is to assign a unique identifier or to determine which index the element is at.

 

Multiple Matches Failure with Recorded or Written Code // How to Wait for Expected Result

Another reason for a multiple matches error is that queries will sometimes not be complete when the test moves onto the next line. A delay should be able to resolve this problem. We tell it to wait for exactly one result (self.count = 1), or wait for an element to exist or not exist (element.exists == true / false) before moving on and give it a timeout value. This is supposed to be fixed for Xcode 7.1.0 however users were again reporting the problem in 7.1.1.

_ = self.expectationForPredicate(NSPredicate(format: "self.count = 1 || element.exists || etc"), evaluatedWithObject: XCUIApplication().tables, handler: nil)
self.waitForExpectationsWithTimeout(5.0, handler: nil)

 

Recorded Code with Special Text/Characters Do Not Compile

Record also may not convert text properly. A button with a unicode identifier was extracted using “\U2192\Ufe0e" which results in a compiler error and had to be manually adjusted to "\u{2192}\u{fe0e}”. Be aware of this for all escaped characters and special texts such as those used in HTML Strings.

 

Record Fails To Capture Action

Record sometimes fails to capture actions. The best way to deal with this was to tap until an action was recorded, then do the action that you need and if it was recorded edit the test after to remove the unneeded actions. Several times I was not able to record an action for one element but was able to record it for the element next to it and then edit the test code to target the correct element.

 

Interacting With Elements Currently Off-Screen

You do not need to include scrolling actions to reach an element far down a page. Once the element is identified it can be interacted with no matter if it is on screen or not; the interaction should bring it on screen. When using Record scrolling actions will be recorded however these lines can be deleted from the test once the element itself is identified.

 

Work Arounds for Accessing Object Properties and States

As you can not access the object’s properties you can not for instance tell if a label is NSString or NSAttributedString (for example to check if text is strikethrough). This is because UITest only uses accessibility attributes and a person using voice-over would not be told text is strikethrough unless it is part of the object’s accessibility traits. Aside from using shared project code (as described above), a work around for this would be to change the item’s accessibilityLabel to be prefixed with “strikethrough” or if in a to-do list “done” would be more descriptive.

Sometimes you need to check that an element's state has changed. You can achieve this usually by checking element.value before and after an action. If it is a label or image you can use the accessibility attributes to have it treated like a button.

 

Nested Collection Views

UITest was unable to identify a collectionViewCell nested in a collectionView nested in a tableViewCell. I was not able to test if a nested tableView has the same problem. It could see the element in the collectionView cell but not the cell itself so state changes on the cell could not be asserted.

 

Animations Causing UI Tests to Fail

In several cases, animation may cause the UI tests to fail (for example, waiting for a button to fade in and be clickable). In some cases, you may be able to wait until the button exists, but in some cases, it is better to disable your animations in your UI tests, and instead use unit testing to check for valid animations.

To disable animations in your test, add the following to your app delegate:

<code>if NSProcessInfo.processInfo().environment["animations"] == "0" {</code>
<code>UIView.setAnimationsEnabled(false)
}</code>

And this to your set up for your UI test:

let app = XCUIApplication()
app.launchEnvironment = ["animations": "0"]
app.launch()

 

Frames For Off-Screen Elements

Queries will return elements which are offscreen. If the frame is needed for test logic or an assertion, before getting the frame make sure it is on screen, usually by tapping on it first.

 

Resources

Apple's UI Testing in XCode
UI Testing in Xcode 7 Part 1: UI Testing Gotchas

iOS9 Day by Day: Day 2: UI Testing
UI Testing Xcode 7
UI Testing Cheat Sheet

 

SEE OUR JOB OPENINGS

Topics: iOS Development, Coding & Development

Abbey Jackson

Written by Abbey Jackson

Abbey Jackson, or Maple Syrup as we like to call her, worked remotely for Metova in 2016.

Enjoy the article? Share it!