• Challenges and Experience in Mobile Testing domain

The mobile market has grown very rapidly in the past few years and the credit can be given to Apple for launching iPhone and then the app store successfully. People are now bending more towards the smart phones because it is making life simpler, now you can access your emails, bank accounts etc easily on phone and no more need to switch on your PC or laptop. However, with the increasing demand, the competition and the involved complexities are also increasing. With the increasing success of Android, many companies were forced to make Android based phone like Motorola, HTC etc which earlier had their own different operating systems. The Android sales for Q@ 2012 went over 50% (https://dottech.org/android/74191/android-tops-smartphone-sales-in-q2-2012-with-over-50-of-sales/). Over 300,000 mobile apps have been devel­oped in last three years and apps have been down­loaded 10.9 bil­lion times.

Mobilization is creating pressure on companies which are into this. With the wide range available in devices, operating systems, carrier providers; it is very difficult to maintain a quality which can sustain across the platforms. The mobile companies are now dwelling deep, analyzing their products, bringing new ventures to compete and most importantly meet the growing demand.

With this comes the need of assuring the good quality of the app, giving rise to the questions like ‘What is mobile Application testing?’, ‘Is the application tested well enough to be launched?’, ‘What are the most basic criteria that it should meet to be successful?’ Mobile application testing has become complex due to the variation available in the devices like wide range in screen resolution, hardware configuration, data speed, camera and image processing. Along with this the Operating systems (iOS, Android, Symbian etc) and the local service providers (GSM, CDMA etc) also add to the complexity. A Map with voice guiding application may work in US but may not work everywhere in India, same way sending SMS may work well in some parts of the world but not in other. All these factors then put an immense pressure on the mobile development and testing teams. It then becomes very difficult to nail down the support criteria.

The Big challenges faced by the Mobile Developers and testers:

       Mobile Devices:

Although I discussed this earlier but let me analyze this more from a developer / tester’s point of view. The major factors (as I said earlier) that affect an application are the screen resolution, the camera size, memory, sound, input modes. When the application designing starts these play a very important role for both developers and testers to decide what the base model should be against which the application is to be designed. This is a really very difficult question to answer as there are people using Galaxy Note, iPhone, tablets etc which if you see closely will have completely different set of configurations.

To solve this big question (as the application which we were working was a Cross platform application which should work on both Android and iOS), we had discussions with our client. The client then did a small survey at his end and in parallel we also ourself looked out for stats. Then the result that we narrowed down to wasn’t a specific model but a range which comprised the iPhone, ipad and LG / Galaxy phone models but no specific model (number).

         Operating Systems:

Currently in market we all know that Android and iOS are the most commonly used platforms while Symbian, Windows are not so popular although they too have their own application stores. For us the requirement was very specific to Android and iOS and hence we were focused on Android and iOS but then the question was what range of versions we should support.

                (http://wmpoweruser.com/wp-content/uploads/2012/07/Q2-2012-US-Smartphone-manufacturers-share.png?e83a2c)

We then checked the version history of Android and iOS, asked the client what version he had, checked for the latest version that we as a team could upgrade the phones that we had. Then finally for our first release of the application we decided to support android 2.3.3 and 2.3.6 while iOS 5.x. We also found that Android 4.x was out in market but wasn’t available for all handsets so we thought of not supporting that for the first release.

  

        Data carrier provider:

There are a lot of local data providers in market offering the best of their services in terms of data connectivity, bandwidth, and speed but still in any case the speed has to be fast. Whether it is a web or native application, both need speed and a good connectivity when the data download or upload is little huge but if the data is very low like in KBs then a low speed might work. The service varies with geography, it may be strong in urban are but very weak in core or the rural parts whether it’s AT&T or Airtel or some other network. Along with this the service may be very strong in US or Canada but less in Asia.

For our application the data download and upload both was little huge in MBs and while testing we needed a very strong wifi range to test. Infact sometimes when the range reduced, the application used to slow down or didn’t work at all. We had assurance from client that in places where they will use this, the data connectivity would be good. Still we tested for no data connectivity and GPRS with a 2G service. We made sure that there were no severe issues if the connectivity was slow or no connectivity.

        Posting the ‘native’ application on the Stores:

When an application specially ‘Native’ is developed and launched; the easiest way to distribute is the application store. You may ask people to download the app in their laptop or PC and then move it to the phone but that may not be very feasible when updates start flowing in. Then taking the application on a store is a big thing because there are some criteria, clauses stated down by Application stores based on which they put the application on the stores. I have even heard that they study the application, its purpose, what benefit will they get like how many people will download everyday etc , what development platform/ programing language is the application built on and then based on all this the decision is made.

For us we have made our application in Titanium (which has some of its own issues), it will be a closed application specific to a company. Hence we are still working out how to make this application go to the stores which is a risk for us. For now we will be distributing the ‘.apk/.ipa’ files to the people and then they will have to take from their PCs to the devices and then install.

    Integration with peripheral devices:

Sometimes there are requirement to have a peripheral device connect with the phone via Bluetooth like the common example is a music application should run well when a person uses Bluetooth headphones. Currently connecting a peripheral device for file transfer via Bluetooth is a common requirement.

We had to make our application take the barcode input from the scanners:

For this we had to enable a Bluetooth connection between the phone and this scanner. We could do this easily for Galaxy S II and iPhone but for a few models like Galaxy S we were not able to establish a Bluetooth connection as these devices did not at all show up in the available devices list. We tried installing an application to enable this connection but that too didn’t work.

These are some of the constrains that are very difficult to overcome specially when there is a very wide range of devices available. 

Usability of the application:

This is one the important challenges which we have to handle very well. UI / usability are the one which appeals a person to use the application. What the font size is, is it easy to tap on buttons, links, etc, is the color scheme used soothing for eyes. All these need to be looked very minutely.


When we had the first look of our app on Android – Galaxy S it was not acceptable because the background was black, the font too small and the images etc very dark. While on iPhone it just looked awesome with the white background. Now making it look good for Android was a challenge for us. To overcome this issue we decided to put same white color background (as of iPhone) for Android as well and it worked.


The other major usability issue we faced was that the app didn’t adjust well in landscape mode and lot of issues started showing up. We spend lot of time figuring out how to resolve this but with titanium it was little difficult so we had to remove the support for landscape mode.

Automation:

When everyday a tester has to run a same set of Suites or cases on an application again and again then the need for automation arises. For mobile application testing, this has been a big question for testers as even today there are not many good tools available in market that can be used across platforms and for native as well as web application. Android and iOS have their own tools, which come bundled with their SDKs like UI Automation and Monkey Runner but then creating and maintaining two different tools is not recommendable.

There are tools available in market like SeeTest (Plug In of QTP), Monkey Talk, etc but out of all these only a very few ones support both platform and that too for both native and web applications.

Four our project we did not need a UI Automation tool but we needed one for Web Service testing. We used SoapUI Pro 4.5 which suited well for our need and is a licensed tool.

Using emulator and simulator:

These are mostly used by developers for development because they provide the same look and feel, functionality as the actual device itself. This is very helpful in testing as well when the testing has to be done on different version of an OS; with emulators and simulator it is very easy to prepare by installing the required version SKDs and using them.

We had to test functionality where we had to track the GPS location of a person signed in the application and send the updated location every 10 minutes. This was very tricky for us as ideally we could not roam around every now and then to test. We tried that also but the locations transmitted very in 7th or 8th decimal place of the latitude or longitude. Hence the solution we used was that, for Android we prepared KML files with variations in the locations and then transmitted them to the application on Emulator via DDMS. While for iOS, the iPhone simulator had its own in built dummy location transmission functionality.

But these have a big drawback that sometimes crashing issues don’t show up. There were issues that continuously kept showing up on the actual device but never showed up on the emulator / simulator. Then the developers had to do a basic testing before release to QA to make sure that there were no crashing issues which they could not see while developing on the Emulator / Simulator.