E-Learning is becoming a driving force of the pedagogical and technological innovations in the higher education institutions worldwide ( Abrami et al., 2008 ; Ruiz et al., 2006 ; Zhang & Nunamaker, 2003 ). Successful e-Learning implementation can increase accessibility, learning flexibility, reduce information overload, improve tracking and lower expenses ( Welsh et al., 2003 ; Garrison, 2011 ).
The most practiced form of the e-Learning in the higher education institutions is blended learning. While there is an ambiguity and many definitions blended learning typically means integrating traditional face-to-face instruction with online instruction ( Ward & La Branche, 2003 ; Garrison & Vaughan, 2008 ). In the blended learning literature, the most common reason, the blended learning is chosen is that blended learning combines “the best of the both worlds”. But if it is not designed well, blended learning environment can also mix the least desirable aspects of the both online and face-to-face instruction.
Blended learning can be implemented in several different levels ( Bonk & Graham, 2006 ) such as activity level, course level, program level and institutional level.
The most common and the most interesting to us is the course level blend. The course level blend involves a combination of distinct face-to-face computer supported activities as part of a course. From Programs in Course Redesign (PCR) effort (The National Center for Academic Transformation, 2006) three blended models have emerged ( Twigg, 2003 ): 1) Supplemental model, 2) Replacement model, 3) Emporium model in addition to the fully online model and buffet model which is large variety of offerings that can be customized to fit the needs of the individual learner..
The Supplemental model
The supplemental model for blended learning incorporates technology into the course, retaining the basic structure of the traditional course. Students may be required to complete online activities, but there is no reduction in course meeting time under the supplemental model.
Under the replacement model, number of in-class meetings is reduced and replaced by out-of-class, online, interactive activities, making fundamental changes to the course. The nature of the in-class activities is changed as well. Instead of traditional lectures, in-class time is freed for more interactive, collaborative learning experiences.
The emporium model eliminates all class meetings and replaces them with a learning resource center, typically a large computer lab, which offers access to course online materials in addition to off-line guidance, making fundamental reformation of the traditional course. Course content is delivered via online materials, and in-person help is provided in the learning resource center.
The researchers point out several reasons to choose blended learning as a learning environment ( Bonk & Graham, 2006 ). Blended learning has a potential to provide pedagogical richness, more access to knowledge, more social interaction, cost effectiveness and ease of revision. A well designed blended learning environment should combine the “the best of the both worlds.”
Leading universities and educational institutions of Mongolia have been implementing e-Learning strategies for the past few years ( Tuul, Banzragch, & Saizmaa, 2016 ). While the e-Learning implementations have been underway for some time, there has been little report on the e-Learning performance in Mongolia.
Gurvan-Erdene University (GEU), one of the pioneering private universities of Mongolia, was established in 1993. About 80% of the graduates of GEU go back to their hometowns, to teach in secondary schools, right after graduation. The teacher major students have to get a teaching license within the first year and renew the license in after 5th and 10th year. In order to achieve teaching license and earn credits, teachers usually have to travel to the capital city or regional centers, which can be time consuming, expensive and even dangerous in bad weather. Mongolia is one of the least densely populated countries in the world with under-developed infrastructure. The cities and town centers are connected by the internet network. While there are disparities in the city and rural areas, average internet connection is 512 Kbps. Increased use of internet is reported for the past years and internet is becoming more affordable ( Ariunaa et al., 2011 ).
According to a World Bank (2010) report, “Mongolia’s tertiary education suffers from low external efficiency, inequitable access, and poor quality.” Blended learning can potentially be used for supporting GEU alumni as well as other teachers working as in-service teachers in remote locations and improve quality in higher education.
Research questions are: Can blended learning
・ Improve the quality in higher education in this setting?
・ Be beneficial for the instructor?
・ Improve learning outcomes?
GEU has been making efforts to implement ICT in teaching in recent years. The college hopes that e-Learning will make positive impact on both faculty and students.
Pilot blended “Pedagogy course” was designed and implemented during 2012-2013 school year then revised in 2014-2015 school year. At the end of the implementation, the students’ blended learning experience survey was completed followed by focus group discussion and statistical analysis.
GEU “Pedagogy course” is a compulsory course for teacher major students and one of the most important courses. The pedagogy course was chosen as a pilot blended course for several reasons: experience of the professor, importance of the course, excellent curriculum and relatively better ICT skills and less course load of the second year students.
Researchers agree that just adding technology to traditional class does not make effective blended learning ( Twigg, 2003 ; Garrison & Kanuka, 2004 ; Derntl & Motschnig-Pitrik, 2005 ), thus whole course redesign, employing learning principles and instructional designs are recommended. To ensure effectiveness, Keller’s ARCS model of motivational design ( Keller, 2010 ), validated by many researches ( Shellnut et al., 1999 ; Chang & Lehman, 2002 ) was chosen for this implementation. The ARCS model considers how to gain attention and ensure persistence during the learning process. The ARCS model is compatible with typical instructional design models such as ADDIE model of instruction ( Molenda, 2003 ) and it can provide useful assistance to designers and instructors, and permits more controlled studies of its critical attributes and areas of effectiveness. There are four steps in instructional design process -Attention, Relevance, Confidence, & Satisfaction (ARCS).
A―Gaining learning ATTENTION strategies
・ Stimulating Perception
・ Give incongruity and conflict
・ Using different types of examples to demonstrate a concept
・ Using humor
・ Using inquiry
・ Requiring participation (Active Learning ? Use of Interactive materials - multimedia)
R―Establishing the RELEVANCE strategies
・ Give lecture―contents that present familiarity (track of course history - user profile tracking)
・ Orient the goals of the students according to lesson or vice versa
・ Show them examples of more experienced students
・ Present the worth of finishing the class
・ Show the future usefulness of the course
・ Match their needs with the course (Learning Style Matching Using Data Mining)
・ Show role models.
C―Building CONFIDENCE strategies
・ Monitoring the learning requirements (Record keeping)
・ Matching the difficulty (Adaptive learning strategies )
・ Knowing the expectation of students and try to satisfy those
・ Defining―announcing the success criteria (Using announcement tools)
・ Presenting tools for goal setting-scheduling (Using calendar-event announce- ments)
・ Providing immediate feedback (Use e-mail- discussion boards)
・ Provide unexpected rewards (such as games)
・ Implement positive outcomes (Give results ? feedbacks immediate)
・ Avoid negative punishments
・ Scheduling―matching the course according to the students expectations
・ Transfer knowledge into real world settings (Use simulations)
・ Be fair in the test results
・ Twigg (2005) recommends the following techniques to ensure quality in blended learning, based on evidences.
・ Continuous assessment and feedback
・ Increased interaction between students
・ Individualized, on-demand support
・ Online tutorials
・ Undergraduate learning assistants
・ Structural supports that ensure student engagement and progress
Clear Goals and standards, appropriate assessment and appropriate workload are also found to be related to quality blended learning experience ( Lizzio et al., 2002 ).
2.1. Pilot Blended “Pedagogy Course” in 2012-2013 School Year
Blended learning “Pedagogy course” was designed based on the existing curriculum and the discussion results with the instructor in charge of the course. ARCS motivational design was employed to ensure quality. Improvements were discussed and made to each chapter throughout the implementation period. Before implementing blended design, typical week started with a face-to-face lecture for the whole 2nd year students, followed with seminars for each groups. Moodle Learning Management System (LMS) ( Moodle, 2016 ) was picked for computer supported sessions.
Use of ATTENTION strategies of ARCS model:
・ Using Moodle forum, the instructor put thought provoking discussion questions, on most topics. Also, as an introduction, a few short stories and essays were put as a Moodle resource.
Use of RELEVANCE strategies of ARCS model:
・ In the beginning the Pedagogy course standard, goals and skills to achieve were introduced and Adobe PDF version was put on Moodle. The face-to- face lectures included not only academic materials but also stories and cases related to school life, family and society.
Use of CONFIDENCE strategies of ARCS model:
・ Goals of each chapter were presented. Instructor provided feedback to discussion
・ forums, essay type questions in quizzes and uploaded assignments.
Use of SATISFACTION strategies of ARCS model:
・ In addition to providing feedback online, face-to-face seminar time was dedicated to case studies by groups and pairs. While the face-to-face feedback was advantageous in immediacy and physical cues, online feedbacks had advantage of more detail and ability to reach each student.
In blended “Pedagogy course,” students are required to review lectures, and prepare for the seminar accessing Moodle server.
Figure 1 shows the basic learning sequence of the pilot blended “Pedagogy course.”
Figure 1. Blended learning sequence of “Pedagogy course”.
In preparation for the blended course, slides and text of lectures, reading materials and other supplementary materials were uploaded. Quizzes are an important aspect of the blended courses ( Spanjers et al., 2015 ). The most time consuming to develop the course was preparation of the quizzes for the course. Each chapter was supplemented with appropriate and randomly selected quizzes. The experiment was conducted in 2012-2013 Fall semester, for 14 weeks.
・ Blended class: 27 students of 2nd year “Primary teacher” major, group A
・ Traditional class: 31 students of 2nd year “Primary teacher” major, group B
Students in Primary teacher major are divided into two groups randomly in the beginning of the first year.
Blended learning experience survey to investigate course design, interaction and learning process was completed and focus group discussion was held in the end of the semester. Blended learning experience survey revealed that the students were highly satisfied with the blended “Pedagogy course.” Open end questions revealed that students were most satisfied with instructor feedback, opportunity to revise assignments and fair grading. Furthermore, opportunity to review and prepare for the face-to-face seminar using online contents, especially quizzes, was highlighted in the students’ feedback. Paper based examination results were compared between traditional and blended classes and blended class students performed slightly better. Instructor feedback revealed that even though there was considerably more work load especially in design stage, the blended class students achieved more skills and some students improved greatly.
Table 1 shows the summary of the pilot blended “Pedagogy course” grouped by ARCS model strategies.
2.2. Revised Blended “Pedagogy Course” in 2014-2015 School Year
Even though the 2012 pilot “Pedagogy course” had a very positive reception by both the students and the instructor, we encountered several problems and there was a room for improvement.
Table 2 shows the revisions made in the blended “Pedagogy course,” grouped by ARCS model strategies.
Table 1. Summary of the pilot blended “Pedagogy course” “+” positive feedback and results, “−” negative feedback and results.
Table 2. Revisions made in blended “Pedagogy course”.
To improve the quality of the blended “Pedagogy course,” while decreasing instructor’s work load, two main indicators, quiz activities and instructor feedback, found in the pilot course were used. First, the number and variety of the quizzes were improved. To reduce the instructor workload, automatic feedback on quizzes were used as much as possible. For example, different feedbacks were provided on each choice of the multiple choice question. Additionally, the seminar hours were reduced by half. Revision and creation of quiz were the most time consuming, increasing work load for both the instructor and the course designer.
In pilot blended course, some students had difficulty using Moodle system even though all first year students have Computer literacy course. Since the implementation of pilot blended “Pedagogy course” revealed that blended class students achieved more skills and to familiarize with blended instruction, blended “Computer literacy course” was taught to all first year students in 2012-2013 school year with very good outcomes.
In revised blended “Pedagogy course,” students are required to review lectures, and prepare for the seminar accessing Moodle server. Figure 2 shows the basic learning sequence of the revised blended “Pedagogy course.”
The second experiment was conducted in 2014-2015 Fall semester, for 14 weeks. The Moodle 2.6 was installed on a portable computer server and hosted at http://3erdene.com/moodle/, with access in the local network at http://192.168.0.1/ in GEU campus. The computer laboratory was scheduled to be used three times a week, for 90 minutes each session. The students could
Figure 2. Blended learning sequence of the revised “Pedagogy course”.
choose which day they come to the laboratory to study. Compared to two years ago, more students have portable computers and smart phones. Unfortunately, most students still don’t have internet connected computers at home. Almost all computer supported activities are expected to be done in the school laboratory. The students were expected to study the lecture notes, complete quizzes and assignments before the face-to-face seminar. Discussions in the forum were also held in some chapters. Chat, news, and reading materials and links were also added. About 70% of Moodle system is translated to Mongolian and the instructions were made for common features of the course.
・ Blended class: 23 students of 2nd year “Primary teacher” major, group A
・ Traditional class: 26 students of 2nd year “Primary teacher” major, group B
Students in Primary teacher major are divided into two groups randomly in the beginning of the first year.
2.2.2. Blended Learning Experience Survey
To assess the students’ blended learning experience, identical survey from the pilot course with 11 quantitative questions with 5-point Likert scale responses (ranging from 1 “I disagree completely” to 5 “I agree completely”) and four open-end questions was used.
The survey was put on Moodle learning page at the end of the blended “Pedagogy course,” with anonymous feedback. Twenty out of twenty three existing students responded to the survey. Responses for quantitative questions and open-end questions of the survey are shown in Table 3 and Table 4 respectively.
Even though the sample size is small, (N = 20, response rate 87%) the results of the study show quite positive response with little variance from the respondents. Compared to the pilot “Pedagogy course,” the responses were less positive in revised “Pedagogy course,” probably firstly due to anonymous nature of the feedback, and because the students took blended “Computer literacy course” first, making this course not something new and exciting like the pilot blended “Pedagogy course”.
Learning experiences: Most satisfied
The items with the most positive responses were similar to the pilot “Pedagogy course”. The students feel that objectives and instructions were clearly stated.
Table 3. Blended learning experience survey quantitative question responses.
Table 4. Blended learning experience survey open-end question responses.
In addition, course structure was perceived relatively clearer in overall experience compared to the pilot course, indicating improved course design.
Learning experiences: Least satisfied
While mostly positive, the students were least satisfied with the system availability the same as the pilot course. Although communication with the instructor may have suffered compared to the pilot course, on the other hand, richness of the content and the course load has relatively improved compared to the pilot course. The response also indicates the improved course design.
Even though the responses imply the improved course design, they still feel that system availability is not enough, and due to reduced seminar hours, communication with the instructor has suffered.
Open-end question responses reveal that the students were very satisfied with the blended learning course. They felt that they reached the learning goals of each chapter. One reason is the opportunity to review the lessons easily, same as the pilot blended “Pedagogy course”. Students felt that the system was easy to use and instructions were clear. In the pilot course the instructor had to give a lot of feedback and that made students extremely happy. In this revised course, with mostly automatic feedback instead of instructor feedback, students were still very happy with the grading. The students felt that the other courses should be taught in blended learning approach, and the majority didn’t anticipate any problems. Reasons for supporting blended learning were: ease of use, opportunities for review, more effectiveness etc.
Focus group discussion
The students feel that there were more opportunities to review and prepare for the seminar. The students feel that blended learning supports the students’ independent study more, and it is easier to collaborate with each other. Students also had opportunities to resubmit the assignments and get feedback from the instructor. Additionally, they could study even when sick or absent etc.
They think that any course can be taught in blended mode, especially the courses that need and encourage independent study such as “Essay writing course”, “English course” etc. The students feel that they have skills to succeed in the subsequent blended courses.
The professor believed that it was easier to teach and there was relatively less work load compared to the pilot blended “Pedagogy course.” She emphasized that the blended class students gained more skills, studied more independently and collaborated more, compared to the traditional class students.
On the whole both the students and the instructor were highly satisfied with course design, social and academic interaction and learning process of the blended “Pedagogy course”.
Based on these results, it can be said that blended learning approach should be used in teacher training and supporting in-service teachers especially preparing for the teaching credential examinations. The teachers can study the materials by themselves, collaborate with other teachers both online and face-to-face and can have support sessions by correspondence or face-to-face when possible. Blended learning would provide more opportunities to study and review and more flexibility for in-service teachers.
2.2.3. Student Performance in Paper-Based Examinations
Next, student performances of blended and traditional classes are compared. Both midterm and final examination were taken by 23 blended learning students and 26 traditional class students. Even though midterm results are similar, blended class final exam mean score is better than the traditional class (see Table 5).
Furthermore, individual student grades show that all but one student improved their grade in the final exam. The study data of the student who worsened in the blended class shows that, although activity numbers are about average, she/he did not submit assignments regularly. Table 6 shows the mean improvement in both blended and traditional class. It shows that students in blended class improved from midterm examinations to final examinations more than the traditional class students.
Moodle study logs were downloaded from the server as Excel sheets after the completion of the blended course and analyzed in Excel 2010 by multiple regression analysis. Table 7, Table 8 and Figure 3 show the regression analysis of Moodle grade average and improvement from midterm to final paper-based exam. The results show that there is some correlation between Moodle achievements and paper-based examination achievements.
The above results show that students in blended class can achieve better results and Moodle study may have positive impact on students’ achievement.
Table 5. Descriptive statistics of the student achievement in Paper-based topic examinations.
Table 6. Descriptive statistics of the student improvement in Paper-based topic examinations.
Table 7. Regression analysis of Moodle grade average and student improvement.
Table 8. Analysis of variance.
Figure 3. Regression of Moodle grades and grade improvement in paper-based exams.
Research questions were: Can blended learning
・ Improve the quality in higher education in this setting?
・ Be beneficial for the instructor?
・ Improve learning outcomes?
The results show that the goal of improving quality of the blended course and reducing the instructor workload has been achieved.
Quality in higher education would help reduce poverty, increase employment, and make better work force for the employment market in developing countries.
I believe that this research not only contributes to the overall research in higher education quality assurance, but also can be used as a guide to improve quality in higher education in other developing countries with similar problems.
Blended learning can potentially be used for supporting GEU alumni as well as other teachers working as in-service teachers in remote locations and improve quality in higher education.
Limitations of the study
Although the research has reached its aims, there were some limitations. First, the population of the experimental group is small, and might not represent the majority of the students. Second, even though the blended learning experience survey shows high level of satisfaction with the blended course, some items have high variance, meaning some participants were quite dissatisfied. Third, even though there are some correlations between Moodle grades and paper based examination improvements, the support could be a little weak due to small number of participants.
Future research should be done with student recommended courses with bigger number of participants in similar settings.
Furthermore, since most of the participants are graduating soon, further study should be done if participating in blended learning class had positive impact for the learning skills and if the experience has any potential to impact their competence as in-service teachers.
This study examined the potentials of blended learning to achieve better quality in higher education. Blended learning class students had high satisfaction because of reasons such as more acquired skills, more independent study and collaboration, more flexibility and more opportunities to review before face-to-face classes. Results show that blended class could achieve better grades than traditional class, with higher proportion of the students with satisfactory grade. Students expressed that they would like more blended courses in the future, despite concerns about laboratory and network environment. The instructor time was used more efficiently and she feels that students acquired more skills in the blended class.
This research result shows that blended learning implementation can improve quality of education, benefiting both students and instructors, becoming positive impact for not only this college but also for higher education and teacher training in Mongolia. These findings show that blended learning can be used for in-service teacher training in Mongolia to achieve better quality and more flexibility for the teachers, especially in remote areas.
Part of this work was supported by Grant-in-Aid for Scientific Research 25280124 and 15H02795.
 Abrami, P. C., Bernard, R., Wade, A., Schmid, R. F., Borokhovski, E., Tamin, R., Peretiat-kowicz, A., et al. (2008). A Review of e-Learning in Canada: A Rough Sketch of the Evidence, Gaps and Promising Directions. Canadian Journal of Learning and Technology/La revue canadienne de l’apprentissageet de la technologie, 32.
 Ariunaa, L., Tsetseg, B., Sayanaa, L., Nurgul, Ch., Munkhbat, A., Odgerel, U., Munkh-Erdene, P., & Khaliunaa, A. (2011). White paper on ICT development Mongolia-2011 (pp. 22-24, 45-46). Ed. Mongolia Interpress.
 Chang, M. M., & Lehman, J. D. (2002).Learning foreign language through an interactive multimedia program: An experimental study on the effects of the relevance component of the ARCS model.CALICO journal, 20(1), 81-98.
 Derntl, M., & Motschnig-Pitrik, R. (2005). The Role of Structure, Patterns, and People in Blended Learning. The Internet and Higher Education, 8, 111-130.
 Garrison, D. R., & Kanuka, H. (2004). Blended Learning: Uncovering Its Transformative Potential in Higher Education. The Internet and Higher Education, 7, 95-105.
 Lizzio, A., Wilson, K., & Simons, R. (2002). University Students’ Perceptions of the Learning Environment and Academic Outcomes: Implications for Theory and Practice. Studies in Higher Education, 27, 27-52.
 Shellnut, B., Knowltion, A., & Savage, T. (1999). Applying the ARCS Model to the Design and Development of Computer-Based Modules for Manufacturing Engineering Courses. Educational Technology Research and Development, 47, 100-110.
 Spanjers, I. A., Konings, K. D., Leppink, J., Verstegen, D. M., de Jong, N., Czabanowska, K., & van Merrienboer, J. J. (2015). The Promised Land of Blended Learning: Quizzes as a Moderator. Educational Research Review, 15, 59-74.
 Tuul, S., Banzragch, O., & Saizmaa, T. (2016). E-Learning in Mongolian Higher Education. The International Review of Research in Open and Distributed Learning, 17. https://doi.org/10.19173/irrodl.v17i2.2227
 Welsh, E. T., Wanberg, C. R., Brown, K. G., & Simmering, M. J. (2003). E-Learning: Emerging Uses, Empirical Results and Future Directions. International Journal of Training and Development, 7, 245-258.
 Zhang, D., & Nunamaker, J. F. (2003). Powering E-Learning in the New Millennium: An Overview of E-Learning and Enabling Technology. Information Systems Frontiers, 5, 207-218.