Mobile Automation with Calabash - An Overview

Mobile Applications

We are currently in the midst of one the biggest software and hardware revolutions. With processing power, storage, and bandwidth increasing exponentially, smart phones and tablets are becoming our main personal computer. As a result customers, employees and other stakeholders are using their smart phones and tablets everywhere which impacts how they see your company online and how they interact with you on a daily basis.

Why to Automate?

Mobile Applications need to be tested frequently and that is why the testing process is becoming time consuming. With Automation, we can write scripts once and can execute it any number of times. This reduces cost, efforts and results in productivity. Also, using Automation we can be sure that our application is stable after every minor/major code change. Normally, Automation Testing can be leveraged even more in Agile development projects. There are lot of tools and framework used for Mobile Automation Testing, Appium and Calabash are the widely used frameworks. Comparing these two frameworks, we found Calabash better than Appium because it doesn’t require any scripting knowledge. It uses Cucumber as a framework and that is why non-technical person can also understand to automate the applications.

Calabash Overview

Calabash is an automated testing Framework for Android and iOS native and hybrid applications. It consists of two test libraries calabash-android and calabash-iOS.

Working of Calabash

Calabash-iOS and Calabash-Android are the two libraries that empower the Cucumber tool to run automated functional scripts on Android/iOS phones and tablets or on simulators. These libraries help QA, developers or domain experts to work at a high level by writing tests in a domain specific language that suite your business domain.

Installation Steps

Calabash installation consists of 2 simple steps, more details are provided on the link below:

http://mobileappautomation.blogspot.in/2013/05/step-by-step-for-calabash-android.html

Benefits of using Calabash

Uniform interface for various platforms: Using Calabash, we have the option of maintaining only one Feature (script) file per app use-case, even if we are developing for both Android and iOS. To overcome cost, companies look for a single interface for different platforms and Calabash provides us with that single interface
Cucumber: Cucumber technology is used in Calabash. Since it uses Cucumber, feature files can be written by QA, domain expert or developers. It provides a way to write the statements either in a simple language or a domain specific language. Using this, it is very simple to understand the feature files or scripts
Devices: Using Calabash, we can run the scripts on physical devices as well as on simulators. We can also run test cases concurrently on various devices
Complex features: Every Test Automation tool has its own unique set of pros and cons. Calabash provides support for complex features such as gestures like swipe, pinch, rotation etc.
Community: There is a large community for Calabash which helps to improve the quality of the product. Using this, we can help Calabash developers to extend the functionality by commenting on their product

Calabash Android Architecture

Whenever we run a feature file, Cucumber provides a mechanism where it creates a test server using which it executes the appropriate feature file. Feature file contains several scenarios which has few step definitions. Finally, the application is accessed and the appropriate feature file is run with scenarios in it.

Android Architecture

Calabash Flow

Following figure shows the design of Feature file. It provides a mechanism where we can specify which objects needs to be called at run time. So we can maintain a single feature file for Android as well as for iOS.

Calabash Flow

Feature file with scenarios

Feature: Login

Scenario: Wrong UserName Login

wait for "Welcome to Android!!" to appear
enter "name" as "username"
enter "password" as "password"
press the "Login" button
wait for "hello" to appear

Scenario: Wrong Password Login

wait for "Welcome to Android!!" to appear
enter "Username" as "username"
enter "word" as "password"
press the "Login" button
wait for "Wrong password" to appear
take picture

Execution Commands

Following commands are used to execute a single feature files or all feature files. We can also run particular scenarios from feature files:

To Run all feature file:calabash-android run <your_apk_file>.apk features/*
To Run a specified feature file: calabash-android run <your_apk_file>.apk features/<feature_1>.feature
Run Specific Tests / Scenarios instead of entire feature file(In case of Multiple scenarios)
Calabash-android run <AndroidApp_debug.apk> --tags @TC_001
To Generate HTML Report calabash-android run <your_apk_file>.apk –format html –out test_report.html

Test reporting

Cucumber Framework provides a way to get the HTML report if something fails while executing scripts. calabash-android run (path-to-apk )–format html –<path_to_report>.html will output the test report in the HTML format.