IIM  Vol.8 No.6 , November 2016
Usability Factors Assessment in Health Information System
Author(s) Majed Alshamari
ABSTRACT
Systems’ usability is one of the critical attribute of any system’s quality. Medical prac-titioners usually encounter usability difficulties while using a health information system like other systems. There are different usability factors, which are expected to influence systems’ usability. Errors preventions, patient safety and privacy are vital usability factors and should not be ignored while developing a health information system. This study is based on a comprehensive analysis of published academic and industrial literature to provide the current status of health information systems’ usability. It also identifies different usability factors such as privacy, errors, design and efficiency. Usability factors are then assessed. Those factors are further examined through a questionnaire to study the priorities of them from medical practitioners’ point of view in Saudi Arabia. The statistical analysis shows that the privacy and errors are very critical than the other usability factors. The study results further revealed that availability and response time are the main challenges faced by the medical practitioners when using the HIS. However, flexibility and customizability were claimed to ease the use of the HIS. In addition, a number of statistical correlations were established. Overall, the study findings seemed helpful to designers and implementers to consider these factors for successful implementation of HIS.

1. Introduction

Presently, usability is considered as one of the main topics in the field of Human Computer Interaction (HCI). Due to its significance and justification, the trend of research has been shifted to this area. Recently, usability is considered as a key quality attribute in the critical systems which is not limited only to e-commerce, games and health information systems. The wide range distribution of health information systems is expected to receive extensive investigations with respect to its assessment, application and adoption. Besides, financial aspect is a key factor should always be considered seriously while assessing a health information system [1] , because 10% - 15% of governments’ annual health budget can be saved by adopting the HIS [2] . Recently the research focused on the usability of HIS from users’ point of view such as doctors, nurses, technicians or even patient [3] . Different usability factors should be considered when focusing on the usability of HIS. Actually, usability questions the level of safety, privacy and error tolerance. [4] predicted that in 20 years from 2002 around half a million new usability professionals might be required. It is well known that usability is not just a single element or characteristic, but there are multiple factors that are responsible for overall usability of a software system [5] [6] . The relationship between certain factors also makes them more or less important in specific applications. For example, data availability has an effect on usability, because when it is analyzed with relation to privacy then it might not have that much importance. Furthermore, usability is more or less dependent on many factors such as efficiency, effectiveness, user’s satisfaction, ease of use and learnability etc. [5] [6] . The importance of usability factors never remain the same for each software system. Like, significance of privacy factors will never be same in a social media website and gaming website. In Healthcare Information Systems (HIS), different elements like time limit of patient visit, availability of patient’s history, analysis of medical reports etc. demand specific factors of usability.

2. Current Issues of HIS Usability

Usability task force, based on Healthcare Information and Management Systems Society (HIMSS) and the Electronic Health Record (EHR) information has specified usability principles for healthcare applications [7] . Among the principles defined by them Simplicity, Naturalness, Consistency, Minimizing Cognitive Load and Effective Information Presentation are significant. Preservation of context, effective use of language, forgiveness and feedback and effective interactions are the other principles that are considered most important by HIMSS for healthcare usability. In a study [8] found that privacy, quality, security and the implementation of electronic medical records are the top IT critical factors that influence Turkish health care systems.

In Saudi Arabia, recent years have shown tremendous increase in number of internet and mobile users. According to a report by Communications and Information Technology Commission [9] , there are 51 million mobile subscribers and more than 16.4 million internet users are in the Kingdom of Saudi Arabia (KSA). The numbers are continuously growing due to easy access and availability of network facilities. [10] conducted a study and showed that, complexity is one of the most significant factors restricting the Saudi Arabian population from adopting an online service. Privacy and compatibility are the other important factors affecting adoption of online services (where compatibility was perceived as similarity with existing values, past experiences and needs). In a similar study, [11] sated that privacy, security, trust, culture and information literacy were the top factors affecting adoption of e-Government services. Generally, practitioners are usually mobile and move frequently inside the hospital. They usually talk to each other and to the patient [7] [12] discussed the usability of labor portal in Saudi Arabia and found that more than 70% of the participants support the argument that the portal is usable. However, the results are mainly from user’s point of view, it needs more experts’ inspection and the usability term needs to be broken down into more reliable attributes. [13] claimed that the usability of a tested mobile application was not satisfactory in Saudi Arabia. [14] found that the digital library interface was not acceptable in terms of its usability based on a sample of 22 undergraduate students. However, this study only investigated the issues related to dynamic web-based systems such as labor portal, digital library interface. However, they clearly stated a number of usability issues such as usability factors that should be taken into consideration. It was found that there is a need to have standards for the electronic health records in various countries such as England, Canada and

Australia. This was found after analyzing the national electronic health records in the above-mentioned countries [15] . However, [16] implies the lack of “Fundamental attributes” to have successful HIS.

In Saudi Arabia, Healthcare Information Technology (HIT) is at the initial stage of adopting the state of the art technologies for this domain. A number of hospitals started using Electronic Health Records (EHR) systems to provide better health services to patients. Moreover, EHRs help HIT professionals in their work [17] and tend to provide assistance in information processing as well [18] . In [19] concluded that, health sector in Saudi Arabia was not ready for implementing successful HIS. In another study, [20] found that 18 factors were seriously influencing the adoption of HIS in Saudi Arabia such as external pressure, network externalities, cost and standards characteristics including the industrial technical standards such as usability and quality.

In [21] identified some factors such as lack of computer skills and technical support that limit the usability and adoption of EHR systems in Saudi Arabia. Recently, [22] studied the rates and levels of EHR adoption in Riyadh, KSA. The study focused on getting feedback from 22 hospitals regarding EHR adoption. The rates and levels were measured from three aspects of EHR adoption i.e., implementation, maintenance and improvement. The results showed that out of 22 hospitals, 11 hospitals were fully implementing the EHR system, while the system adoption was in progress in 8 hospitals and the remaining 3 hospitals did not adopted any system. Factors having relationship with healthcare system adoption are hospital size (positively related), ownership of hospital (private or government) and somewhat type of development team. In [23] , it has been identified and presented six barriers such as human barriers, professional, technical, organizational, financial and legal barriers for the adoption of healthcare systems in Saudi Arabia. This research clearly identified that human factors were at top of the list of barriers in adoption of healthcare systems in KSA. Usability of health information system is a critical factor for patient safety. Having a usable system can significantly help the clinicians to spend extra time with their patents [24] . Error number can also be reduced if the usability of the HIT is maximized.

3. Study Objectives

4. Research Methodology

A quantitative research approach was adopted in this study. A questionnaire was developed as the research methodology. The questionnaire was designed by following the usability factors described by many investigators [7] [25] [26] to achieve the study objectives. However, the literature review was the main source for building the usability factors. The questionnaire for the survey was distributed manually to 250 practitioners in 5 different hospitals in Saudi Arabia. The answers to the questionnaire were based on a five-point scale for assessing the usability factors, where the users rated the importance of the usability factors from a ten-point scale. A pilot study was conducted using the preliminary research questionnaire on a small number of professionals in health care sector in Saudi Arabia. In addition to that, two usability experts were consulted in order to avoid any misunderstanding or ambiguity or vagueness in the survey. The pilot study helped the researcher to receive constructive feedback in order to better formulate the questions. Then, the final questionnaire was distributed after taking into account all the comments from the professionals and the experts. The analysis of collected data from the final questionnaire was presented. The data was divided into three sections namely survey response background, assessing and prioritizing the usability factors and the qualitative analysis with respect the importance of the usability factors. The various statistical techniques such as ANOVA and regression were applied for data analysis according to procedures given by [27] .

5. Results and Discussion

The data collected from the respondents during the survey was analyzed and presented in Table 1. The total response was around 57%. Out of a total of 250 questionnaires, only 143 completed questionnaires were received which corresponds to 65 (45%) male

Table 1. Respondents background and details.

Table 2. Usability factors and reliability tests.

*Reliability for usability factors assessment; **Reliability for usability factors priorities.

Table 3. Descriptive analysis for all the items.

Min: Minimum, Max: Maximum, SE: Standard error.

system quickly, preventing doctors from making mistakes, patient data privacy and the privilege to browse the data with total score of 7.8, 7.9, 7.9 and 8.1, respectively. However, the less important items in this study were related more to efficiency such as performing the task more quickly, improving the efficiency of the users, the need to remember a lot of things while performing a task and using familiar medical terms with a total score of 6.9, 6.8, 6.8 and 6.9, respectively. Although, there is a slight difference between the top and the less important items, yet it was noticed that items related to privacy and errors received high importance than those related to efficiency. A strong relationship was found between privacy and errors elements using the Pearson correlation test with a p-value less than 0.05. Also, the correlation between learnability and efficiency elements was also statistically strong as the p-value was less than 0.05.

Figure 1 shows the ranking for the usability factors from the participants’ point of view. They ranked the privacy as the top important factor with a score more than 8. This finding is in line with other research in healthcare carried by [8] who stated that, the privacy factor scored the top position where the stakeholders have some concerns about. He also rated all the IT factors that may influence HIS in Turkey. Errors factor was also considered as a high important factor with a score value of 7.89. It was found that clear error messages are not available in the HIS when evaluated according to the findings of [24] . These results clearly conflict with the usability guidelines, in particular to heuristics, proposed by [4] . Errors and problems in the system or in the network were also ranked as one of the top technical issues in HIS in Saudi Arabia [23] . Although, he referred to more technical problems than usability problems as these issues affect the system usability either directly or indirectly. The practitioners did not rank design factor as a high important factor. These results agree with the finding of [29] , who reported that the practitioners did not consider design as an important factor because it is not important to them to see good UI design. Another interesting finding was reported by [26] who stated that all the physicians usually spent more time to enter the patient data. This finding endorsed the results of this study as the efficiency and effectiveness were evaluated as less important usability factors from their point of view with a score value of 7.02 and 7.15, respectively. It is difficult to argue that efficiency

Figure 1. User’s rating of usability factors by different participants for the adoption of HIS in Saudi Arabia.

and effectiveness are less important usability factors as they scored more than 7 out of 10. This can justify the importance of all the usability factors used in this research, whereas, data in Figure 2 show the usability factors assessment by the different participants. It can be seen from the results in Figure 2 that learnability and efficiency were the top ranked factors with a value of 3.8 and 3.72, respectively as judged by the participants. These were followed in descending order by privacy (3.63) > satisfaction (3.56) > design (3.56) > effectiveness (3.5) > errors (3.38).

Moreover, the participants had the opportunity to add more usability factors that are important and would like to see in the health information systems. Also, they might have been asked about the necessary modifications in order to have a usable health information system. They emphasized the importance of having a flexible and customise system as one of the participants strongly commented “The system should allow us to edit/modify a report that is saved” as they are not allowed to edit any data after they save it. They also mentioned that the importance should be simple so that they can interact easily with the system without any duplicate entries to go for a certain task or job. Besides, confirmation is needed by the system authority where duplicate data is required. The interesting finding in this study was that out of total participants, 5 participants clearly commented on the need for a specific functionality from the system such as medical reports, statistical reports and the specialties should be separated from each other to improve the usability of the system they usually use.

The critical problems that reduce the easy use of the system were also investigated in this research. A total of 8 participants stated that the response time is a critical problem for them. They mentioned that “The system is really slow”. This result was in line with those of [23] , who concluded that system slowness was one of the top technical issues. In another study, [29] mentioned that response time was an important finding where the practitioners do expect the system to response quickly [29] [30] .

It was clear that the availability is one of the critical problems faced by the participants. Moreover, it is not mainly a usability problem unless linked with a feature such as archiving or patient history data is stated clearly. In addition, error recovery and completeness were also among those critical usability problems. These prevent them

Figure 2. User’s evaluation of usability factors by different participants for HIS adoption in Saudi Arabia.

from undoing a mistake they made or if they wanted to view a patient record. Because, they do not prefer to have a paper file with the system as it delays the work and distributes them as they claimed. In his study [21] stated that, the HIS system was difficult to use due to its complexity. These results should help developers to overcome the above-mentioned issued and to address them in any proposed HIS. Taking into account such usability factors will lead to successful implementation in the future.

6. Conclusion

Usability of health information system is a critical factor for patient safety. It can improve the effectiveness and efficiency of a system. Having a usable system can significantly help the practitioners to spend extra time with their patents. This study explored the current issues with respect to HIS with a focus on Saudi Arabia. The results of this study for assessment of usability factors can help designers and implementers for successful implementation of HIS. This study also showed as how the practitioners prioritize the usability factors. Privacy and errors were ranked as the top usability factors which should be considered while designing and implementing HIS. Response time and availability were also reported as additional critical factors and should be seriously considered. In addition to these findings, usability factors showed strong correlation among its elements such as privacy and errors. In conclusion, this study tried to convince the decision makers for adopting the HIS by realizing the importance of usability factors and to present as how the health professionals see the current HIS.

Cite this paper
Alshamari, M. (2016) Usability Factors Assessment in Health Information System. Intelligent Information Management, 8, 170-180. doi: 10.4236/iim.2016.86012.
References
[1]   Hamborg, K.C., Vehse, B. and Bludau, H.B. (2004) Questionnaire Based Usability Evaluation of Hospital Information Systems. Electron. J. Inform. Syst. Eval., 7, 21-30.

[2]   Alkraiji, A. (2011) Issues of the Adoption of HIT Related Standards at the Decision-Making Stage of Six Tertiary Healthcare Organizations in Saudi Arabia. Ph.D. Thesis, University of Loughborough, Loughborough.

[3]   Ozok, A.A., Wu, H., Garrido, M., Pronovost, P.J. and Gurses, A.P. (2014) Usability and Perceived Usefulness of Personal Health Records for Preventive Health Care: A Case Study Focusing on Patient’s and Primary Care Provider’s Perspectives. Applied Ergonomics, 45, 613-628.
https://doi.org/10.1016/j.apergo.2013.09.005

[4]   Nielsen, J. (2002) Becoming a Usability Professional.
http://www.nngroup.com/articles/becoming-a-usability-professional/

[5]   ISO 9241-11 (1998) Ergonomic Requirements for Office Work with Visual Display Terminals (VDTs). Part 11: Guidance on Usability. International Organization for Standardization, Geneva.

[6]   Nielsen, J. (2003) Usability 101: Introduction to Usability.
http://www.ingenieriasimple.com/usabilidad/IntroToUsability.pdf

[7]   Jeffery, L.B., Grayson, R. and Barnes, J. (2009) Defining and Testing EMR Usability: Principles and Proposed Methods of EMR Usability Evaluation and Rating. Technical Report, Healthcare Information and Management Systems Society (HIMSS), USA.

[8]   Turan, T.H. and Palvia, P.C. (2014) Critical Information Technology Issues in Turkish Healthcare. Information & Management, 51, 57-68.
https://doi.org/10.1016/j.im.2013.09.007

[9]   CITC (2013) Communications and Information Technology Commission. Annual Report, Kingdom of Saudi Arabia.

[10]   Al-Ghaith, W.A., Sanzogni, L. and Sandhu, K. (2010) Factors Influencing the Adoption and Usage of Online Services in Saudi Arabia. Electron. J. Inform. Syst. Dev. Countries, 40, 1-32.

[11]   Alateyah, S.A., Crowder, R.M. and Wills, G.B. (2013) Factors Influencing Citizen Intention to Adopt Egovernment in Saudi Arabia. Proceedings of the International Conference on Information Society, Toronto, 24-26 June 2013, 250-255.

[12]   Al-Zuabi, H.F. and Al Shaikhli, I.F. (2013) Evaluation of Usability Problems of Labor Portal in Saudi Arabia. International Journal of Advanced Computer Science and Information Technology, 2, 14-25.
https://doi.org/10.2139/ssrn.2372783

[13]   Alqahtani, M.A., Alhadreti, O., AlRoobaea, R.S. and Mayhew, P.J. (2015) Investigation into the Impact of the Usability Factor on the Acceptance of Mobile Transactions: Empirical Study in Saudi Arabia. International Journal of Human–Computer Interaction, 6, 1-35.

[14]   Alasem, A.N. (2013) Evaluating the Usability of Saudi Digital Library’s Interface (SDL). Proceedings of the World Congress on Engineering and Computer Science, 1, 178-181.

[15]   Deutsch, E., Duftschmid, G. and Dorda, W. (2010) Critical Areas of National Electronic Health Record Programs: Is Our Focus Correct? International Journal of Medical Informatics, 79, 211-222.
https://doi.org/10.1016/j.ijmedinf.2009.12.002

[16]   Altuwaijri, M.M. (2008) Electronic-Health in Saudi Arabia. Just around the Corner? Saudi Medical Journal, 29, 171-178.

[17]   Likourezos, A., Chalfin, D.B., Murphy, D.G., Sommer, B., Darcy, K. and Davidson, S.J. (2004) Physician and Nurse Satisfaction with an Electronic Medical Record System. Journal of Emergency Medicine, 27, 419-424.
https://doi.org/10.1016/j.jemermed.2004.03.019

[18]   Margalit, R.S., Roter, D., Dunevant, M.A., Larson, S. and Reis, S. (2006) Electronic Medical Record Use and Physician-Patient Communication: An Observational Study of Israeli Primary Care Encounters. Patient Education and Counseling, 61, 134-141.
https://doi.org/10.1016/j.pec.2005.03.004

[19]   Al-Solbi, A. and Mayhew, P. (2005) Measuring E-Readiness Assessment in Saudi Organisations Preliminary Results from a Survey Study. Proceedings of the 1st European Conference on Mobile Government, Brighton, 10-12 July 2005, 467-474.

[20]   Alkraiji, A., Jackson, T. and Murray, I. (2011) Health Data Standards and Adoption Process: Preliminary Findings of a Qualitative Study in Saudi Arabia. Campus-Wide Information Systems, 28, 345-359.
https://doi.org/10.1108/10650741111181616

[21]   Khudair, A.A. (2008) Electronic Health Records: Saudi Physician’s Perspective. Proceedings of the 5th IET International Seminar on Appropriate Healthcare Technologies for Developing Countries, London, 21-22 May 2008, 38-44.
https://doi.org/10.1049/ic:20080575

[22]   Aldosari, B. (2014) Rates, Levels and Determinants of Electronic Health Record System Adoption: A Study of Hospitals in Riyadh, Saudi Arabia. International Journal of Medical Informatics, 83, 330-342.
https://doi.org/10.1016/j.ijmedinf.2014.01.006

[23]   Khalifa, M. (2013) Barriers to Health Information Systems and Electronic Medical Records Implementation. A Field Study of Saudi Arabian Hospitals. Procedia Computer Science, 21, 335-342.
https://doi.org/10.1016/j.procs.2013.09.044

[24]   Pereira, R., Duarte, J., Salazar, M., Santos, M., Abelha, A. and Machado, J. (2012) Usability of an Electronic Health Record. Proceedings of the IEEE International Conference on Industrial Engineering and Engineering Management, Hong Kong, 10-13 December 2012, 1568-1572.
https://doi.org/10.1109/ieem.2012.6838010

[25]   Villa, L.B. and Cabezas, I. (2014) A Review on Usability Features for Designing Electronic Health Records. Proceedings of the IEEE 16th International Conference on E-Health Networking, Applications and Services, Natal, 15-18 October 2014, 49-54.
https://doi.org/10.1109/healthcom.2014.7001812

[26]   Goldberg, L., Lide, B., Lowry, S., Massett, H.A., O’Connell, T., et al. (2011) Usability and Accessibility in Consumer Health Informatics: Current Trends and Future Challenges. American Journal of Preventive Medicine, 40, S187-S197.
https://doi.org/10.1016/j.amepre.2011.01.009

[27]   SAS (2010) Base SAS 9.2 Procedures Guide: Statistical Procedures. 3rd Edition, SAS Institute Inc., Cary, 17-34.

[28]   Dewberry, C. (2004) Statistical Methods for Organizational Research: Theory and Practice. Routledge, New York, 364.
https://doi.org/10.4324/9780203414897

[29]   Shabot, M.M. (2004) Ten Commandments for Implementing Clinical Information Systems. Proceedings (Baylor University. Medical Center), 17, 265-269.

[30]   Boonstra, A. and Broekhuis, M. (2010) Barriers to the Acceptance of Electronic Medical Records by Physicians from Systematic Review to Taxonomy and Interventions. BMC Health Services Research, 10, 231.
https://doi.org/10.1186/1472-6963-10-231

 
 
Top