Preparing environment

To start the following process you need to locate the APK file you want to test. The APK file could be downloaded from the web or get it direct from the development team.

Then read the instructions to install the APKTool available in the following link, you can create a working folder and allocate there both files: apktool and apktool.jar. Once you have configured your working folder withe APKTool, execute the following instruction:

java -jar apktool.jar d PackageName.apk -o TargetFolderName

The process should finish like this:

I: Using Apktool 2.2.2 on PackageName.apk
I: Loading resource table...
I: Decoding AndroidManifest.xml with resources...
I: Loading resource table from file: /Users/rcervantes/Library/apktool/framework/1.apk
I: Regular manifest package...
I: Decoding file-resources...
I: Decoding values */* XMLs...
I: Baksmaling classes.dex...
I: Copying assets and libs...
I: Copying unknown files...
I: Copying original files...

At this point you are ready to dig in the files extracted. Why is important this step? because you can identify how the application was developed, which technology is using, if it’s native or web mobile application. The folder PackageName.apk contains useful information about the application that allow us to prepare an strategy for UI Testing.

Next, prepare you APK to be tested with Calabash, for this step you need to install and configure Ruby in your environment, remember that Calabash is based in Ruby and Cucumber, so it’s important have configured our development machine with the most latest version of Ruby.

It’s time to create the ruby workspace for our application, in the working folder write:

bundle init

If Ruby environment it’s correctly configured a new file with the name: Gemfile will appear in the working folder. It’s time to add the following gems that will be needed to execute Cucumber, Calabash-Android and Xamarin Test Cloud.

# frozen_string_literal: true
source ""

gem 'calabash-android'
gem 'cucumber'
gem 'xamarin-test-cloud'

Once added the gems, it’s time to install them.

bundle install

The process should finish like this:

Resolving dependencies...
Using awesome_print 1.7.0
Using builder 3.2.3
Using bundler 1.13.6
Using gherkin 4.1.1
Using cucumber-wire 0.0.1
Using diff-lcs 1.3
Using multi_json 1.12.1
Using multi_test 0.1.2
Using escape 0.0.4
Using httpclient 2.8.3
Using json 1.8.6
Using retriable 2.0.2
Using thor 0.19.4
Using rubyzip 1.2.1
Using mime-types 2.99.3
Using cucumber-core 1.5.0
Using luffa 2.0.0
Using xamarin-test-cloud 2.1.2
Using cucumber 2.4.0
Using slowhandcuke 0.0.3
Using calabash-android 0.9.0
Bundle complete! 3 Gemfile dependencies, 21 gems now installed.
Use `bundle show [gemname]` to see where a bundled gem is installed.

Once all gems has been updated it’s time for configure the test environment for Calabash-Android.

calabash-android gen

The process should finish like this:

I'm about to create a subdirectory called features.
features will contain all your calabash tests.
Please hit return to confirm that's what you want.

features subdirectory created. 

At this moment, a new folder (features) has been created, this folder is used to save the features and the steps used for the test cases in Calabash-Android.

step_definitions (folder)
support (folder)

It’s time to build our UI test case.

my_first.feature file represents the stack of scenarios and steps made in Cucumber for behaviour-driven development that will be executed by Calabash-Android. For instance:

Feature: Login Module
  Scenario: Login user
  Scenario: New user

It’s time to drill down inside the application, Calabash-Android allow to us to see the native metadata objects generated in each view, in the working folder execute:

calabash-android console PackageName.apk

The process should finish like this:

No test server found for this combination of app and calabash version. Recreating test server.
Done signing the test server. Moved it to test_servers/2f8c19fcb2edf9cd78e2891bede41693_0.9.0.apk
Starting calabash-android console...
Loading /Users/rcervantes/.rbenv/versions/2.3.3/lib/ruby/gems/2.3.0/gems/calabash-android-0.9.0/irbrc
Running irb...
  • In some scenarios the APK file should be resigned to be tested, this can be performed in the working folder with:
calabash-android resign PackageName.apk 

It’s time to reinstall the application and start the test server.

irb(main):001:0> reinstall_apps
irb(main):002:0> start_test_server_in_background
  • It’s important verify that the Android emulator is running to host our test environment.

Let’s query all objects in the application, this can be performed in the working folder with:


Queries will help us to create the steps necessary to be used by Cucumber in the features.

For instance, steps.rb file can contain:

require 'calabash-android/calabash_steps'

Given(/^The app has launched$/) do
  wait_for do

Then (/^I should write my login-username with the text "([^\"]*)"$/) do |username|
    wait_for_elements_exist(["CustomEditText index:0"], :timeout => 20)
    enter_text("CustomEditText index:0", username)

Then (/^I should write my login-password with the text "([^\"]*)"$/) do |password|
    wait_for_elements_exist(["CustomEditText index:1"], :timeout => 20)
    enter_text("CustomEditText index:1", password)

Then(/^I should press login button$/) do
    wait_for_elements_exist(["CustomButtonView index:2"], :timeout => 20)
    touch(query("CustomButtonView index:2"))

And can be called by my_first.feature

Feature: Login Module

  Scenario: Login user
    The app has launched
    Then I should write my login-username with the text ""
    Then I go back
    Then I should write my login-password with the text "myPassword"
    Then I go back
    Then I should press login button

Once the features and steps scripts are locally tested, it’s time to send it to Xamarin Test Cloud.

test-cloud submit PackageName.apk [API Key] --devices [DevicesGroupId] --series "master" --locale "en_US" --user

More information: