Mobile application development and testing process

Recently, we have been involved in mobile application development and testing. The below checklist ensures that both developers and testers have covered these high level scenarios during their requirements discussion, development and testing activities. Mobile application development and testing checklist also helps you refine your requirements to ensure that your scope of work is clearly defined. These are high level questions and not very specific to the application functionality (we will cover that in the next article in the series).




1. Which mobile platform to develop for? iOS or Android?
iOS and Android are the preferred platform for developing mobile applications. However, Blackberry is still used by several enterprise users and a significant population of the developing world still uses Symbian phones. It’s good to know upfront which platforms are expected to be supported. This will help you make crucial decisions regarding your architecture / design and also provide inputs on whether you want to develop a completely native application or use a framework like PhoneGap. Popular platforms are listed below:








Android
iOS
Windows Mobile
Blackberry
Symbian
Mobile application development and testing smartphone market share

Mobile Application Testing Device Units Forecast

This Forbes article provides details on the current market share and forecast, based on mobile OS.

2. Which version of iOS should I target? What version of Android should I develop for?
Does your application use any features introduced in a specific version of the OS? If so, you will want to mention this in your marketing material and also test it on the target OS. It is also good practice to prevent the application from being installed on OS versions that are not supported. This will avoid users leaving low ratings and negative feedback after installing the application on an unsupported OS. Of course, you will want to test and ensure that your application works on the targeted OS version.

Mobile application developmentmobile application testing iOS Version chart

The above image shows the usage statistics for different version of the Android and iOS. Google provides updated metrics for the version of Android being used across all android devices and information on how you can support multiple versions. Apple also provides statistics for iOS version usage along with a checklist of their own (These stats are from Sept 2013 before iOS7 launched).

3. Device Hardware Requirements
Does your application have any specific hardware requirements like memory, camera, CPUs etc? As mentioned above, it’s best to prevent installation on unsupported devices programmatically when possible. Here are instructions to check if your device has sufficient memory/RAM in Android and iOS and also to check for camera on iOS and Android.

4. Which screen resolution should I target?
Make sure that your application looks good on your target screen resolution. Smartphones and tablets come in all shapes and sizes. A list of devices with screen resolution and display density is available on Wikipedia. Some of the common screen resolutions are below:

320 x 480px
640 x 960px
480 x 800px
720 x 1280px
768 x 1280px
800 x 1280px
1200 x 1920px
2048 × 1536px
Google also provides statistics on the number of devices that have a particular physical screen size and density. Information on pixel count for various screen sizes and how you can support multiple screen sizes is also available.

5. Should I develop another app for tablets?
It’s good practice to use high quality graphics for large devices like tablets especially if your application or game is expected to be used on these devices. Some developers release a separate HD version of the application/game instead of using a single package. Irrespective of your implementation process, it’s good practice to test on both devices if you expect significant users on both. Google has also released an app quality checklist to help developers deliver quality apps for tablet devices.

6. Portrait or Landscape Orientation?
Some games work only in landscape mode while some applications are designed to work in portrait mode only and other work in both modes. Make sure you test your applications to see if there are any issues when changing the orientation like application crashing or UI bugs.

7. Testing GPS functionality, Accelerometer, Hardware keys
If your application requires use of the following hardware features, your test cases also need to test for scenarios when they are not available -

Hardware keys – Ex. Camera application using a dedicated camera button, Task/Event Manager applications using hardware buttons to snooze a reminder, media players using volume and other keys etc. Some applications also use the power button to provide additional functionality / shortcuts to application behavior.
Accelerometer – Applications that make use of accelerometer require testing to ensure that the readings are being recorded accurately and utilized correctly within the applications. This test case might be relevant to applications like Star Maps, Pedometers,  Jump trackers, Games, 3D visualization applications etc
GPS – How will your Navigation applications respond if the GPS is disabled or turned off abruptly during operation?
Any other sensor – If you application depends on additional sensors for temperature, luminosity or any accessory that provides additional functionality, then you need to ensure that you have tested for conditions when they are not available or do not function accurately.
8. Network Connectivity Issues – GPRS, 2G, 3G, WiFi, Intermittent connectivity, No connectivity
Most of the applications are developed in the presence of WiFi which provides good network connectivity. However it’s important to test applications in the real world where the user might not have access to WiFi. Usually when people are on the move, network connectivity is intermittent with connection being dropped once in a while. Network speeds also vary based on the users location and the kind of connectivity they are paying for. Applications must be able to handle these situations gracefully and they must be tested for it.

9. Test Mobile + web app updates
Does your mobile application have a server side component or a web service it uses? Does the mobile application need an update when the server side component is updated? If so, make sure there is a test case to track this to avoid any human error.

10. Testing interruptions to the mobile app
There are various events that can interrupt the flow of your application. Your application should be able to handle these and should be tested for the same.

Incoming Call
Text message
Other app notifications
Storage low
Battery low
Battery dead
No storage
Airplane mode
Intermittent connectivity
Home screen jump
Sleep mode
11. Mobile application security testing
Security and data privacy are of utmost importance in today’s scenario. Users are worried about their data and credentials being exposed through vulnerable applications.

Is your application storing payment information or credit card details?
Does your application use secure network protocols?
Can they be switched to insecure ones?
Does the application ask for more permissions than it needs?
Does your application use certificates?
Does your application use a Device ID as an identifier?
Does your application require a user to be authenticated before they are allowed to access their data?
Is there a maximum number of login attempts before they are locked out?
Applications should encrypt user name and passwords when authenticating the user over a network. One way to test security related scenarios is to route your mobile’s data through a proxy server like OWASP Zed Attack Proxy and look for vulnerabilities.

12. Testing In-app payment, advertisements and payment gateway integrations
If your app makes use of in-app payment, advertisements or payment gateways for e-commerce transactions, you will need to test the functionality end to end to ensure that there are no issues in the transactions. Testing for payment gateway integration and advertisements will need accounts to be created with the Payment Gateways and Advertisement servers before testing can begin.

13. Mobile application performance testing
Have you checked to see if the performance of your mobile application degrades with increase in the – size of mailbox, album, messages, music or any other content relevant to the application?

It’s good practice to test your application for performance and scalability issues. With large storage capacity being available at affordable prices, it’s not uncommon for users to have large amount of data / content on their smartphone. Users even store SMS for several years on their smartphones. If your application has user generated content / data associated with it (Ex. Photographs, SMS etc) which can grow to huge proportions over the lifetime of the application, your testing should include these scenarios to see how the application performs. In case the application has a server side component, you should also test the application with increasing number of users. While this testing can be done manually, we have tools like Little Eye and Neo Load that can help you with Performance and Load testing of your mobile application.

14. Mobile Application Localization and Timezone issues
If your application is multilingual, it needs to be tested in other languages to ensure that there is no character encoding issue, data truncation issue or any UI issues due to varying character lengths. You also need to test applications to ensure that they handle timezone changes. What happens if a user travels forward across timezone and returns to his/her previous timezone? How does your app handle entries with date and time which are in sequence but not in chronological order?

15. Testing Social network integration
Many applications these days ship with the ability to share a post from the application, on the users’ social networking account. However most users would like to be prompted before a post is published on their account. Does your application handle this? Are they being allowed to share the status message being shared?







16. Test hardware connectivity – Bluetooth, WiFi, NFC, USB – Device recognition
Smartphones come with a plethora of connectivity options. If your application makes use of the below connectivity options (Ex. File managers or photo editors that let you share your file, AirDroid which allows you to transfer files between PC and your mobile over wifi) then you should test them to ensure they work as expected. You should also test to see how they handle errors when the connection is lost during a transfer / transaction. Commonly used mechanism to share data or transact are:

Bluetooth
WiFi
USB
NFC
17. Google Play / Apple App store integration and supported device list/restrictions
Consultants and organizations that provide end to end services should also include test cases to ensure that the mobile app is successfully deployed to the App store / Play store and is only available to the supported devices. This could also include validation of all the text, screenshots, version numbers etc that are part of the app listing.

In the next article in this series we will cover test cases related to mobile application functionality.

If you found this post useful, please share it with your friends / colleagues.