Journal of Software Engineering and Applications
Vol.07 No.12(2014), Article ID:51900,9 pages

Smart Phone Application Evaluation with Usability Testing Approach

Naseer Ahmad, Muhammad Waqas Boota, Abdul Hye Masoom

Department of Computer Science, Virtual University of Pakistan, Lahore, Pakistan


Copyright © 2014 by authors and Scientific Research Publishing Inc.

This work is licensed under the Creative Commons Attribution International License (CC BY).

Received 23 September 2014; revised 18 October 2014; accepted 13 November 2014


In this paper, we present a narrative role methodology based on different people’s opinions, which is now implemented in our research environment for the purpose of testing novel techniques, the usability testing of most common used smart phones such as Android and Apple smart phone application. This research analysis is carried out for various corresponding features to compare or evaluate different features on the basis of modern smart phones’ operating system, framework for hardware and software, battery life and many more features. Usability testing of smart phones’ software application is a promising research context that nowadays faces a number of challenges because of sole features of mobile phones, narrow bandwidth, varying environmental factors and unreliability of wireless connection or networks. A number of questionings are applied to accumulate user’s opinion about ongoing features. Users of Android and Apple smart phones reply by answering based on their routine life usage experience. In this paper, we will also elaborate and provide a new proposed model for mobile devices.


Usability, Smart Phone, Laboratory, Field, Android, iPhone

1. Introduction

The usability testing of smart phone application is mainly carried out by the most general tool usability testing in a development practice. To test smart phone application with usability testing tool, tests are commonly carried out with the help of think aloud protocol, which is based on Ericsson and Simon’s work. In this protocol particular tasks are assigned to users in the test domain and the users are encouraged to think loudly when they are trying to perform their specific tasks. With this approach the information required to us in which user’s interface corresponds the natural human way of thinking, performing and emphasizes the characteristics and process for improvement.

Traditionally, usability tests to analyze smart phone applications were carried out in test laboratories. For example, usability test laboratory was consisting of a common living room, or an office attached to a monitoring area through a one sided mirror. The environment of usability test laboratory is a peaceful environment in which the usability tester can only focus on his/her specific task. The practitioners and usability testers think that it is a problem to perform evaluation of mobile phones in laboratory. Therefore, they do not suggest the domain where smart phones are used and be short of the desired environmental strength. The contiguous environment and mobility are assumed to a group particular requirement for smart phones applications. The usability testing approach must fulfill these requirements.

In smart phone a small video camera, source of mobile video recording has swiftly developed during the last few years and mobile users perform their user tests to test this feature in the field. In the field usability testing to test smart phone applications becomes much simple and straightforward. Users of smart phones are able to record the screen of smart phones while attaching a mini camera to it and obtain information for later evaluation. This permits the users to test corresponding tests in the field as tests performed in the laboratory, and the test leader is able to follow what is occurring on the smart phone’s screen and also able to hear users’ comments. This condition also permits the usage of think aloud protocol in smart phone’s usability test in the field, even though the useful tools testing development in the field is still probable to be extra time consuming as compared with the laboratory setting [1] . Usually, the researchers follow members while walking and request them to perform on specific prepared tasks [2] . Videotaping members during their walk at stores with conventional cameras may be much disturbing and can experience unusual things.

Numbers of studies have evaluated the tradeoffs among laboratory and field usability testing evaluation [1] [3] -[5] . Their main goal is to explore whether field usability testing is significant to the mobile application assessment, the additional time and efforts. Therefore, usability testing approach must be selected suspiciously when usability testers want to plan a usability evaluation of a smart phone application. In some situations, filed tests are performed better than laboratory tests, and vice versa. For example, usability testing in field is particularly better when there is no effect of environment such as noisy environment while travelling on the train and talking on smart phones during travelling via train. However, Kjeldskov et al. and Kaikkonen et al. [1] [5] recognize laboratory usability testing as working well as compared with the field usability testing, while Duh et al. and Nielsen et al. [3] [4] disagree with these results.

In the beginning connections of traditional mobile phone were made by using their specific model to make safe and secure communication over the network. The novelty performance and technology provided us to find the high stage level that smart phone devices currently have the competence to run desktop applications. In order to fill up the situation of such development for mobile devices, hardware of such mobile devices automatically has being acclimatized to control these changes like long battery life. In the modern period of life smart phones benefit from the implementation of successive features like touch screen and multitasking. Nevertheless, the mechanism of implementation totally remains diverse from one mobile to another mobile, which is distinguished by the approach that is arranged to satisfy the users’ requirements. Globally, users are really interested and active users of smart phones also get pleasure from the advantages of applying applications developed by the smart phone’s software developers. The development of most excellent products, applications, design reengineered approach and online stores’ competencies construct the maximum degree of competition among different smart phone companies. Android smart phones have additional applications. Additionally, Apple has concentrate on the architectural design reengineered approach and online stories.

As smart phones are growing rapidly, variation among smart phones and dumb phones is doubtful. In general, operating system and touch screen are those features of dumb phones that are most common features of smart phones. In the end, the research describes that smart phones operate on an open operating system and lastingly connect to the Internet. There are a large number of users of smart phones such as Android and Apple’s phones, who benefit from their application developed by the software developers.

Since there is higher level of competition among different companies of smart phones, so these companies are launching different products especially with different design style, applications and online stores to explore attraction towards smart phone lovers. Consequently, Apple has focused on the design styles and online stores; on the other hand, Android has focused on developing addition applications. Figure 1 presents a framework to test smart phones in laboratory and field. Emulator is used for usability test in laboratory for smart phones.

In Part 2 we will discuss related work, in Part 3 we will discuss usability testing of smart phone applications, in Part 4 we will discuss research methodology, in Part 5 we will describe the discussion and proposed model. Finally in Part 6 we conclude our research work.

Figure 1. A structure for design and implementation of usability testing of smart phone applications.

2. Related Work

Traditionally, usability test were carried out for the intention of evaluation of general usability of user interface of Wireless Access Protocol (WAP), portal accessed with mobile phone, and such portal provides access to number of usual mobile services like news, e-mail and leisure information. All such services can be discussed anywhere, anytime, afford the user is in the context of coverage of mobile phone operator. There is no precondition essential in order to access particular service, and any user who has a mobile phone well-matched with WAP technology may be assumed a special user.

Different techniques were described to perform the mobile usability tests based on the evaluation framework. These frameworks diverged only in three phases: device used for steering on WAP portal such as mobile phone, or emulator, location or environment for performing the usability test such as laboratory or field, and device used in order to register the communication such as computer or wireless camera. The remaining elements of evaluation framework like tasks, measures, criteria for choice of members participated and metrics of usability were exactly the alike for these techniques.

3. Usability Testing of Smart Phone Applications

As we know that system has significant role by developing novel technologies, which support the users in number of different contexts. For evaluation of corresponding technologies usability testing is conducted, and it is associated with the developing system properties, features and system interface. In usability testing of smart phone application, the unique features such as smart phone context, connectivity and smart phone screen size, different display resolutions and restricted processing capability and power.

3.1. Smart Phone Context

The context of mobile phone can be described as “any information that describes the condition that associated with communication among users, applications, and neighboring environment” [6] . It usually contains the location, individualities of close people, environmental components, and object, which can divert user’s attention. It is not easy to choose an approach that can contain all opportunities of smart phone context in a unique usability test.

3.2. Connectivity

Not too good and reasonable wireless network connection in smart phones with minimum bandwidth is a general issue for smart phone applications. While performing the usability test of smart phone application this key attribute has its own importance, because it is not good for data downloading and value of streaming data such as video streaming. Data transfer rate along with its signal strength in a wireless connection can differ at number of time, surroundings, and mixed by the user mobility. Hence, the questions is that how we can solve this issue, and it should be taken into consideration while performing the smart phone application’s usability test.

3.3. Smart Phone Screen Size

The screen size of mobile phone or device that is the physical constraint has a great effect on the usability of mobile phone. Through appearance of number of web pages on a small screen phone may be aesthetically not emerging, unavailable, and not in good case, completely unreadable.

3.4. Different Display Resolutions

Usually, the range of display resolution of smart phones is much minimum than personal desktop computers. The range of smart phones resolution normally is 240*320 pixels, and this low range of display resolution of smart phones degrade the feature of multimedia information of a smart phone. Therefore, different smart phones with different display resolutions may vary the usability test results that may be conducted in the field or laboratory.

3.5. Restricted Process Capability and Power

The memory capability and computational potential of smart phones lag far personal computers. Numbers of applications do not sustain low memory such as graphic hold up, rapid processing speed, like 3D application to get city map for personal digital assistants, which cannot be practical for such smart phones. Since, the proces- sing capability is restricted of smart phones, so the application developers are bounded to disable unrelated functions.

4. Research Methodology

It is very challenging to evaluate two different most emerging products in the business market. On the behalf of product’s evaluation, one product is better because of its generous, and other one is better because it fulfills its user’s requirements. To the meet user’s requirements, we conduct a survey and perform usability testing evaluation of most popular smart phones such as Android and Apple iPhone.

4.1. Setting Goals

Before conducting our survey we study approximately all quality attributes of both Android and iPhone that are useful to develop narrative model for smart phones. During our survey we evaluate the most key features of both leading smart phones, and organize into the five main types to build a new emerging model of smart phones. The features that we organize into one proposed model are operating system, executing applications, multitasking support, integrated technology, and compatible framework. Our proposed model is almost useful and compatible with the user’s requirements.

4.2. Data Gathering Interviewing Process

A variety of data sources are applied during our survey, in which interviews, literatures, and observations are included. In interview process with current users of Android and iPhone, we ask different key questions to them related to their running smart phones. We perform our survey for research point of view in the industry to activate the data to steer the interview process. During our research work we complete our work in two stages, in the first stage we obtain the relevant features of both Android and iPhone, and we conduct a questionnaire in the interview process. In the second stage, we conduct another questionnaire on the basis of quality attributes of both products, and we organize the emerging features of both Android and iPhone into one group to form a better model for smart phones. We also get feedback from current end users, and communicate with skilful people of relevant context to get helpful ideas. Interviews are conducted with end users of both Android and Apple iPhone, experts, and industrialists. In our survey we get feedback from total 64 persons, in whom 32 persons were end users of Android, and 32 persons were existing users of Apple iPhone, or familiar with both these leading products.

4.3. Evaluation of Research Study

Evaluation of our research study is carried out on the basis of usability testing of these products. We gave questionnaire to 64 persons and after this we make observations. The feedback obtained from these persons is given in Table 1.

4.4. Empirical Work

Usability testing discusses various attributes from users’ perspective. Initially, usability research method is assumed as it helps to reduce product’s cost and effective approach to collect the data from different users while in relation to the usability test, it is logically not easy for researchers to do their research of usability testing because of time, mandatory resources, user’s refusal usability testing constraints. Our conducted survey for usability evaluation of smart phones contains rating scale for evaluation. In our research proposal all questions were based on the usability testing criteria, and 62 end users relevant to the particular applications in android and Apple iPhone were chosen randomly for evaluation point of view.

4.5. Findings

The objective of our survey is to evaluate Android and Apple iPhone applications, and to offer a new proposed model for Android and Apple iPhone application into one integrated emerging application. Usability testing attributes evaluation is given in graphical representations. In both Android and iPhone usability evaluation while interview users approximately do not show strongly disagree their opinion while asking all questions related to specific smart phone.

In the following graphical representation the values are in percentage in Y-AXIS and Z-AXIX represents the attributes of both Android and Apple iPhone (Figures 2-5).

5. Discussion and Proposed Model

After evaluation of smart phones applications through usability testing, we will discuss about new model of smart phones that how new model after integrating the features of Android and Apple iPhone is much effective than original model.

5.1. Integrated Technology in Smart Phone Devices

In any device the integrated technology is relevant to the architecture of a particular system. In our proposed model, the developed system incorporates various technologies of network management that also contain the software design of Android smart phone. The strange attribute of such system is also effective as Home Gateway Entertainment (HGI), Portable Network Device (PND) unit and Vehicle On-Board Unit (OBU). It also has a charming feature for users that it offers a suitable wired and wireless connectivity. Two functions are offered by our proposed model such as roaming and sharing. The interface of such model is prepared by various network interfaces. In wireless connectivity roaming offer useful and effective services to choose and search emerging resources such as RSS, accurate bandwidth for user’s requirement based on the particular situation. In various but identical network roaming is much useful. However, the sharing capability of new system has the

Table 1. Feedback from 64 persons.

Figure 2. Responses toward strongly agreement.

Figure 3. Responses toward agreement.

Figure 4. Responses toward neutrality.

Figure 5. Responses toward disagreement.

ability to assign resources to the users of such system (Figure 6).

Users of smart phones alike usually come across sharing capability, particularly at home or in large plazas and isolated place. The roaming functionality makes use of intellectual mobile internet device when smart phones’ users are out of their home and buildings. The user’s demands can be fulfilled when this integrated model make use of available resources.

5.2. Supported Running Features and Application

The Android system development has the capability to manage the content units that are used for the explanation of managed feature that execute on Android application. A programming software context such as Java Native Interface (JNI) executes on JVM that usually, represent two features of functions: to call and to be called by the JNI, which is written in the C/C++ assembly language. Android device also needs JNI interface for a particular application. It is required in essential situation that the developers of Android application must have an alternative to use the Android NDK, surrounded in the code local libraries like the desired platform; then application executing on JVM in JNI interface may be used to call local function. A significant large module in the local code that is written in the C/C++ language can be developed by the reuse interface of JNI.

5.3. Richness of Operating System

Android operating system development is applied on the hardware platform, and here we discuss the detailed

Figure 6. A Structure for design and implementation of usability testing of smart phone applications.

features of such operating system. Linux is an essential part for development of Android operating system, and suppose we have Linux fedora and gcc 4.3.2 version, JDK, Python, and Repo to build Android operating system. According to the operating system requirement, Android mobile system is configured at X86 system architecture. A significant feature is the development of the Kernel of Android smart phone based on the latest version of Linux. This helpful and emerging because it will provide us memory management capability, process management, security and many other benefits.

It is almost necessary to run peripheral of smart phone that usually helps to execute Android operating system on the target device. The source of Kernel previously contains drivers, and selects the utilities that also compatible with drivers, and few driver dependent utilities like power management, touch screen, protocol stacks, and network utilities insert and change the right driver source which truly forces the target device. The 802.11 wireless LAN cards are much suitable for collocating tools software drivers, and need for configuration tools as well. In order to make kernel image after resource configuration, the compilation of kernel files is carried out.

The Android smart phone system basic library relies on its fully capable operating hub such as media support, exterior management, and database engine. Certain utilities such as QoS, and handover are usually implemented by the defined libraries by every end users and it has ability to improve the network connection services that is also has the feature of significant quality resources of network that depends on the user’s services. The system of Android smart phone and its network utilities has the demand of external tools for the purpose of its integration, and such utilities are also associated with the application and kernel layer, and this is suitable for the file system of Android based smart phones, and this file system contains (busy box command, wireless configuration, network, and dialler) tool files. The application framework utility’s source code in which resource manager, windows manager, and lot of other utilities needs modification.

A representative function like curs is incorporated to application framework utility in the proposed model, and the purpose is to make capable the Android system much appropriate for distant managing by end users. Software Development Kit (SDK) is an emerging tool for this Android based application. Internet in another way is also much effective for the purpose of to get data by using the wireless device. Therefore, on such discussed factors the services like call handover to other base station is assumed more significant and it improves the communication quality over the network. The operating system of Android smart phones has been fruitfully inaugurated to the target device as a result of developed application.

5.4. Multitasking and Multimode Support

The convenient laboratory system development of Android smart phone devices has the help of multitasking and multimode, and key requirement of such lab system is to be followed in our proposed model that maintains the design and technologies used in such model. The launched standards and requirements about our proposed approach for new model for smart phones will carry the following principles:

· The creation of user accounts and validation for industrialists and academicians containing business people, students, and professors;

· Smart phone application connection to the remote server;

· Withdrawal or retrieval of date from server with the help of data acquirement board that should contain a database to carry the collected data;

· It should be necessary that server is connected to the proposed model’s application of smart phone;

· The system of information imitation of smart phone from database server, the option of online/offline made promising, and should be controlled by the remote database;

· Remote and native database should be synchronized collectively;

· The visualized information should be at vocal command and read that make use of various data and time criteria;

· It should have the capability to explain and be queried on data readings that may visualize particular data explanations;

· It should be installed in the Android framework.

5.5. Framework with Current Applications of Google & Microsoft

It is essential that compatibility in all current framework that the Android based applications are managed with all current applications such as Google and Microsoft. In general, Android is needed to offer multimedia framework software support stack, and such visualization of Android is essential to integrate all software components, algorithms, and their device drivers. It will manage the multimedia application meeting principle that can simply transplant. As excellent performance is required, so hypothesis of small power embedded multimedia support, and large computational load of playback might be understand. Digital Signal Processor (DSP) is also required to compute the multimedia data. Additionally, an essential demand of our new model is codec for encoding and decoding the audio and video. So, multimedia framework requires form codec.

6. Conclusion

The usability testing evaluation of users’ products has much importance to launch new better technology than the current one. In this paper, we discuss two usability testing approaches of mobile phones such as field test and laboratory test, and explain how field test and laboratory are carried out in different environments. We carry out surveys based on the features and quality attributes of two most charming smart phone products Android and Apple iPhone to evaluate their usability while collecting views of their end users and show this evaluation in tabular and graphical representation. We also provide an integrated model of both Android and Apple iPhone in a unique model. The proposed model comes into existence by merging features of both Android and IPhone into a single architecture. After usability test survey, we become competent to find out the main differences among these two leading products in the market of smart phones. Additionally, we provide a next generation model of smart phones. On the behalf of evaluation of these smart phones, we offer a much leading process to introduce an ideal smart phone with lots of additional features. In the end we find out that Android has more emerging characteristics than IPhone. It will be better to integrate the most charming applications of Android with iPhone to make an ideal smart phone for their end users.


We are thankful to Dr. Ghulam Ahmad Farrukh who gives us an opportunity to do research work. We are also thankful to our friends, colleagues and participants in the interviews to give their feedback.


  1. Kjeldskov, J., Skov, M.B., Als, B.S. and Høegh, R.T. (2004) Is It Worth the Hassle? Exploring the Added Value of Evaluating the Usability of Context-Aware Mobile Systems in the Field. Mobile Human-Computer Interaction-Mobile HCI, 2004, 61-73.
  2. Ballard, B. (2007) Designing the Mobile User Experience. John Wiley & Sons, Chichester, 173-174.
  3. Nielsen, C.M, Overgaard, M., Pedersen, M.B., Stage, J. and Stenild, S. (2006) It’s Worth the Hassle! The Added Value of Evaluating the Usability of Mobile Systems in the Field. Proceedings of the 4th Nordic Conference on Human- Computer Interaction, New York, 14 October 2006, 272-280.
  4. Duh, H.B., Tan, G.C.B. and Chen, V.H. (2006) Usability Evaluation for Mobile Device: A Comparison of Laboratory and Field Tests. Proceedings of the 8th Conference on Human-Computer Interaction with Mobile Devices and Services, New York, 2006, 181-186.
  5. Kaikkonen, A., Kallio, T., Kekäläinen, A., Kankainen, A. and Cankar, M. (2005) Usability Testing of Mobile Applications: A Comparison between Laboratory and Field Testing. Journals of Usability Studies, 1, 1931-3357.
  6. Dey, A.K., Salber, D. and Abowd, G.D. (2001) A Conceptual Framework and a Toolkit for Supporting the Rapid Prototyping of Context-Aware Applications. Human-Computer Interaction, 16, 2-4.