After considerable growth over the last two decades in online learning, graduate programs show a steady increase in the number of available online course offerings. Graduate students are attracted to online programs for many reasons. According to Allen and Seaman (2008), lower total costs, including commuting and relocation, are invaluable for many students. Because many course materials are often available online for free, the need for textbooks and their associated costs are both lessened. Additionally, non-monetary burdens such as traffic and limited available time are all pearls of e-learning that may make traditional classroom settings pale in comparison.
As technology advances, so does the design, content and delivery of online education. Approximately 66% of universities’ chief academic leaders agree that online learning is essential to the universities’ long-term strategies(Allen & Seaman, 2014). However, there is limited research delving into what the best pedagogical practices should be for adoption by graduate-level online programs. Brick and mortar institutions worldwide offering conventional face-to-face instruction have long outlined their best teaching practices in graduate education. These best pedagogical and learning practices include providing resources for smooth functioning and optimal interactions between students and faculty; encouragement for faculty and students to form scholarly communities; diversity of race, gender and social class; fostering students to publish papers; intellectual forums for exchange of ideas; and adequate mentoring by practitioners in the field. Many of these practices are employed in universities’ online programs, however, the unique facets of online education suggest that there should be other best practices that are more specific to e-learning settings. Yet, the evolution of best practices for the digital mode of delivery appears to be lagging behind. With the increased prevalence of graduate-level online programs, the use of the related best pedagogical practices and guidelines will inevitably aid educational institutions, as well as faculty and students within them. The identification, evaluation and adoption of e-learning best practices are of strategic importance to educators. As a result of this and the dearth of published work in this area, we sought to conduct a systematic review related to best pedagogical practices in graduate-level online learning. The overall aim of this review was to answer the following question: What are the best pedagogical practices in graduate-level online learning? The primary objectives were to:
・ Prepare an answerable Population, Intervention, Comparison, Outcome, Time (PICOT1) question and a well-formulated SR protocol,
・ Examine and determine pedagogical practices that are most effective in promulgating timely degree completion for online graduate students, and
・ Organize, evaluate, analyze, and synthesize the data for manuscript submission to a peer-reviewed journal for publication.
The PICOT question for this systematic review was “In graduate level online learning, which pedagogical practices are associated with achieving the best educational outcomes for graduate level online students?” Within the PICOT framework, the population is defined as graduate-level online students, the intervention is defined as pedagogical practices in online learning and the outcome is defined as best educational outcomes. The comparison and time are not applicable.
2. Study Characteristics
Throughout the researched literature, many of the studies’ populations shared similar characteristics, while other studies varied in their populations. There were multiple population variations in the studies we reviewed; the majority of the studies’ populations were graduate-level online students. Some studies included both graduate-level online students and faculty; followed by both undergraduate and graduate students. Few studies had all three student populations: undergraduate, graduate and doctorate students. Other studies included the populations of only graduate level online faculty or doctorate students. Though some of these studies included undergraduate-level online students, we were able to use the data collected for only graduate-level online students and other professional-level students in our systematic review. This is outlined in the Evidence Table(Appendix B), which highlights the details of the included studies.
As with the population characteristics, there were also similarities and variations of setting, evidenced by the types of courses and programs used in the included studies. The majority of studies did not specify the type of course; those that did specify included, education, seminary, and Black studies courses. One setting was the College of Liberal Arts while other settings consisted of Master of Business Administration (MBA), Doctoral of Education Technology and Master of Arts in Teaching. Population characteristics are presented in greater detail in Appendix B.
3. Review of Literature
Several studies have examined teaching practices in online graduate learning in an effort to increase student retention, graduation rates and student engagement. A detailed outline of studies can be found in the Evidence Table(Appendix B). In our systematic review, we reviewed a total of twenty-four studies and each yielded different results as to what are the best pedagogical practices in graduate-level online learning. One study was conducted by Evans (2013), consisting of a survey, interviews, and grade comparisons of MBA students at a private middle states university to measure students’ perception of learning effectiveness in face-to-face versus online MBA courses. Another study we reviewed was by Groeling (2004)that was conducted to find if technology impacts communication and student learning in face-to-face and online courses. Groeling (2004)further investigated student pre-existing motives, interaction, interpersonal communication competence and mediated communication competence to learn and find the impact of instructional technology on cognitive learning outcomes. Groeling (2004)found that there is a lack of interaction between instructors and students.
A study completed by Hamrick (2004)at 17 different universities evaluated the perceptions of satisfaction and motivation in students’ overall online class experiences using a Web-based survey instrument. A study by Hegngi (1997)examined and described the patterns of the first iteration of the Media Technology and Diversity course with a focused analysis of its online interaction. Both Hamrick (2004)and Hegngi (1997)discovered there is a lack of the interaction between students and instructors, along with a lack of effectiveness regarding the course contents.
In addition, we reviewed a study by Henckell (2007)conducted at a midwestern institution in Missouri. The purpose of the study was to help institutions by providing information that assisted them in developing a distance education system, by creating a proper student evaluation instrument. Henckell (2007)found that there is a lack of student involvement in instructors’ pedagogy practices. In another study, Holland (2006)observed online threaded discussion forums used for learning in higher education. Holland (2006)revealed that there is a lack of different ways to enhance students’ ability to learn better. Researcher Hwu (2011)conducted a study using mixed methods to investigate concerns and professional development needs of instructors at the University Alaska Fairbanks (UAF) on the adoption of online learning. Hwu (2011)demonstrated that there is a lack of support for instructors in online learning. We reviewed a study by Ice (2006), which examined the lack of technical support and pedagogical guidance provided to faculty and student satisfaction in online courses. Another study by Kahrhoff (2005)investigated the relationship between the non-academic support needs of online graduate students and their developmental achievement. Kahrhoff (2005)discovered a lack of support services available to online students. Kim (2015)explored student motivation of online graduate students at a Korean university medical school, finding that when there was a deficiency in instructor-student interaction, the retention rates declined. After studying graduate students enrolled in a fully online instructional setting at a Canadian university, Kovanovic, Gasevic, Joksimovic, Hatala and Adesome (2015)emphasized that lower quality of the online activities and assignments resulted in poor cognitive development and lower cognitive presence. Similarly, Kuna (2012)found that the less frequent interaction graduate students had within online course management, the lower the performance. This observation was most apparent in threaded discussions.
Levinson (2012)studied graduate teacher candidates in an online environment to measure successful implementation of learned lessons and strategies. The findings suggested that the lack of support from instructors, described as receiving little to no feedback, prevented the candidates from using strategies learned from their online courses. After surveying 401 graduate students over two semesters, Merena (2006)demonstrated through student surveys that inadequate performance evaluation and feedback decreased student confidence in their learning achievements. A study reviewed by Marchand and Gutierrez (2012)explored what role the lack of emotional student support played in the graduate online environment. A study by Mozzani-Miller, P. (2006)examined the quality of learning through student engagement in two types of distance education classes in the same institution, which was evaluated using quantitative data from each class in the form of the four dependent variables: the length of the student responses, the number of sources cited, the number of interactions elicited by the response to questions and the number of times sources outside the virtual library were cited (p. 123). Another study by Pattison (2017)investigated to what extent the social presence of online faculty influence and serve as a successful strategy to increase academic achievement, student satisfaction, and persistence in online courses, highlighting the need for strategies improve student grades, satisfaction, or persistence of student success. Randle (2013)analyzed interactions and outcomes associated with an online professional development course for science teachers and found that lack of explicit framing of learning tasks in an online environment appears to have a great deal of influence on learning outcomes. Ruhlandt (2010)examined the differences in retention, social presence, cognitive presence, and teaching presence in fully online and blended courses. Ruhlandt (2010)emphasized that students that adhered less to the aforementioned strategies were more negatively impacted and less successful. Ruiz (2007)analyzed student satisfaction in distance education students and conventional-campus-based students from the same graduate education program and found campus students to be less satisfied then their online education counterparts. Lastly, after revealing the prevalence of insufficient technical support and training, Wang (2007)highlighted specific needs for adequate socialization and technical support of students, particularly for first-year, online graduate students.
In summary, multiple studies throughout the review of literature highlighted that the interaction between students and instructors had the greatest perceptual impact on the students. When students received delayed feedback from instructors, it generated student perceptions of dissatisfaction and isolation, thus, making a timely student-instructor interaction a key factor in student learning. This systematic literature review included 24 studies dated from 1997 to 2017, with a data time gap from 1998 to 2004. The lack of quality studies on the best pedagogical practices in online learning during that period highlighted a need for additional, up to date and more extensive research. This systematic review delivers additional research and provides insight on the subject that has been understudied despite the considerable growth in online learning. Additionally, this systematic review outlines the practical significance of the findings for not only students, but for faculty and institutions of higher education, as demonstrated in Table 1.
4.1. Eligibility Criteria
4.1.1. Types of Studies
Our primary studies did not include other systematic reviews in order to help avoid double counting studies included in both systematic reviews and to reduce the likelihood of forming inferences and conclusions biased by the results of reviews done by others. Although we did include mixed methods studies in this systematic review, qualitative studies were excluded, primarily because they rely more on less formal and subjective methods to collect data and, thus, heavily depend on the skill of the researcher. Also, the results of qualitative studies can be easily influenced by personal bias(Occupytheory.org, 2014). Qualitative studies also tend to lack the rigidity that systematic reviews aim to achieve. In the scientific community, evidence-based practices are crucial and qualitative studies provide a summary or overview of a topic(SJSU, 2017). Therefore, we included mixed methods studies, viewing the qualitative part of the mixed study as a supplemental overview material, and only quantitative portions were included in the data tables.
4.1.2. Types of Outcome Measures
We included outcome measures from studies discussing current and past pedagogical practices implemented in both graduate-level online and traditional graduate classroom-based settings to determine the best pedagogical practices for online graduate learning. We screened the data, which resulted in 132 full-text articles that were assessed for eligibility. After initial review, we analyzed 42 full-text articles with risk of bias assessed, and excluded 18 articles with reasons. Due to the rigorous selection process and strict systematic review guidelines, we selected and included 24 studies in this narrative systematic review. The research study selection process was documented in a Preferred Reporting Items for Systematic Reviews and Meta-Analysis, or PRISMA, flow diagram(Figure 1). We considered the outlined specific outcomes to determine the best pedagogical practices for online graduate learning:
・ Student retention rates
・ Competency with technology
・ Teacher professional development
・ Timely degree completion, student engagement
・ Development of enhanced online educational approaches
・ Enhancement of student satisfaction and motivation
4.1.3. Information Sources
The search included published journal articles with a date range from 1995-2017, higher education learning settings only, and primarily English results. We mainly used the Rutgers University Libraries, specifically the George F. Smith Library of Health Sciences. Through the Rutgers University Libraries, the databases used in the search included Ovid MEDLINE, Web of Science, Cumulative Index to Nursing and Allied Health Literature (CINAHL), Educational Resources Information Center (ERIC), Pro Quest Dissertations & Theses Global and Scopus. Additional searches of grey literature resources such as reports, dissertations, theses, conference proceedings, studies outside of the United States, and studies conducted in other languages, given accurate and available translations, also occurred. We separated and categorized literature resources as grey or non-grey, and included grey literature in the data tables.
4.1.4. Search Strategy
The core of our search strategy consisted of identifying the major elements of
Figure 1. PRISMA Flow Diagram (summarizes the selection process).
our PICOT question and translating natural language terms to subject descriptors, or Medical Subject Headings (MeSH) terms. MeSH terms included graduate programs, e-learning, teaching methods, graduate education and online learning. We also searched different combinations of terms. We conducted a search of any potential unpublished studies, white papers and other grey literature relevant to this review.
4.1.5. Time Period
Research over the specified time period of 1995 to 2017 allowed us to capture the literature about the use of technology in higher education, both in its infancy and at its height(Rajasingham, 2011).
4.2. Study Records
4.2.1. Data Management
EndNote was used by the team of authors for bibliographic management.
4.2.2. Selection Process
Three review authors independently screened the titles and abstracts and applied inclusion and exclusion criteria according to the research article title. Articles were independently screened again to review the titles and/or abstracts for consensus. In order to ensure a widely accepted and systematic approach was used, we adhered to the PRISMA flow chart(Figure 1).
4.2.3. Data Collection Process
After the articles were screened, we conducted a full text review of the remaining included articles and independently evaluated each article to determine if it meets eligibility for inclusion. A thorough examination was performed of the study design and studies that explored the use of pedagogical practices in graduate-level online learning and included them in this review. The data in this review is comprised of the secondary data extracted from the eligible articles. Additionally, all authors of the systematic review were available to provide necessary additional information, clarification and further evaluation of included studies, as needed.
The review authors independently extracted the following information from the studies and documented it on the data extraction form, which was developed based on Cochrane Collaboration’s Data collection form:
1) Study Characteristics: study aim, study design, study setting, inclusion criteria, and exclusion criteria;
2) Study Participants: this systematic review study did not include any participants;
3) Methods: methods for assessing bias and methods for extracting, combining and synthesizing data;
4) Interventions: characteristics of possible interventions were discussed;
5) Outcome Measures: description of main outcome measures;
6) Results: organization, evaluation, analysis, and synthesis of data collected.
4.2.4. Data Items
In order to collect all the relevant information about the best pedagogical practices in graduate-level online learning, we researched, combined, and analyzed the study data using narrative analysis.
4.2.5. Outcomes and Prioritization
The studies were grouped according to study design and prioritization was given to studies with higher quality of study.
4.2.6. Risk of Bias in Individual Studies
Aside from addressing risk of bias through researcher consensus during times of disagreement, we also used the Cochrane risk of bias criteria to address the risk. This criterion is widely used for judging the risk of bias, which includes: random sequence generation, allocation concealment, blinding of outcome assessment, incomplete outcome data, selective reporting, or other sources of bias. Risk of bias was rated as low, high, or unclear for each study being considered for inclusion. Studies with high or unclear ratings were excluded.
4.2.7. Addressing Missing Data
Missing data was not a reason for study exclusion from the review. If we were unsuccessful in obtaining missing data, such as in the case of Virginia Tech Conference, we assessed the risk of bias appropriately and coded the study as incomplete outcome data. We discussed the potential implications of the absence of the study from this systematic review.
4.2.8. Data Synthesis and Reporting
The data extracted from the studies was entered and grouped into a data extraction form as well as an evidence table (Appendix B) according to study design. Most importantly, we evaluated and reported the findings in the context of an ensemble of recommendations related to best pedagogical practices for online graduate learning. These recommendations were further supported by the outcome measures in the studies (Appendix A).
4.2.9. Confidence in Cumulative Evidence
We assessed the strength of the body of evidence according to the GRADE criteria where quality of studies varied between very low, low, moderate and high.
5.1. Description of Included papers
The initial database search strategy resulted in 1014 findings. We assessed full-text articles for eligibility and twenty-four papers were considered relevant for this systematic review(Figure 1). A full list of the included studies (n = 24) is found in Appendix B.
The majority of studies were survey-based, followed by mixed-methods, followed by two Quasi-experimental studies, one cross-sectional, one literature review, one instructional based, two correlational studies, one systematic research, and one casual comparative. The most frequently used measures were learning effectiveness and evaluation of online learning. This was followed by two studies that measured online student’s satisfaction, two studies that measured student engagement in online learning, two studies that measured online instructors training and development and two studies that measured social presence in online learning. There was a study that measured online graduate students’ development and support needs, another study that measured learning technology, an additional study that measured best graduate online course design and a study that measured best strategies to reduce dropout rates in online courses. The above information is referenced with greater detail in Appendix B.
5.3. Outcome Measures
Outcome measures included in this review were from studies in which researchers evaluated current and past pedagogical practices implemented in both graduate-level online and traditional classroom-based settings to determine the best pedagogical practices for online graduate learning. Specific outcomes, as outlined above in Section 4.1.2, were considered in order to determine the best pedagogical practices for online graduate learning. Numerous studies used certain tools to measure such outcomes, such as student enrollment trends and data to measure student retention rates, as well as student surveys and questionnaires to measure student engagement.
5.4. Best Pedagogical Practices Found
Based on the reviewed literature and relevant studies, the best pedagogical practices associated with achieving the greatest educational outcomes for graduate-level online students were having positive academic discipline and attitude towards online education, and clear, timely and consistent teacher-student communication. The importance of quality of the student-teacher communication was the most noted and apparent throughout the literature. Also, studies including Henckell (2007)and Mozzani-Miller, P. (2006)showed better educational outcomes in an online setting when instructors conducted a pre-course characteristic-type evaluation of the student, targeting demographics, computer competency, distance education experience and preferred method of learning. This allowed instructors to adapt their pedagogy to the students enrolled to identify and work with at-risk online learners, and at the same time, enhance student satisfaction and motivation. Post-course assessment of graduate-level online students included questions that covered course content, materials used, organizational aspects of course activities, technical aspects, and the performance of instructor and support staff to help identify possible future course improvements. Mozzani-Miller, P. (2006)also recommended to “use multiple means of quality of learning assessment developed specifically for distance education courses, such as developing and employing pre-test and post-test questionnaires for online learning students to discover: pre-test―demographics, computer experience, distance education experience, preferred method of learning and post-test―student satisfaction, grades and gain scores” (p. 126). Mozzani-Miller, P. (2006)also reported statistical analysis revealed that by the end of the course all of the students achieved a significant gain in quality of learning. Pertaining to course activities, Hegngi (1997)and Hamrick (2004)found that posting all of the assignments all at once and allowing students to work on their own pace along with not having the burden of responding to every single student’s post in the class would be better teaching practices.
Increasing student retention in the online graduate setting was one of the major outcomes of this systematic review. Studies including Groeling (2004), Hamrick (2004), Hegngi (1997), and Henckell (2007)showed that student interaction with the instructor, clear and steady communication between faculty and online graduate students, as well as prompt and constructive instructor feedback, all factored into a student’s satisfaction, engagement and overall success in the online distance education environment. Ruhlandt (2010)also examined the importance of student retention, social presence, cognitive presence, and teaching presence in fully online and blended courses. Some of the strategies to reduce such differences included student orientation, readiness identification, technical support (24 hours a day) and personal contact with professors, other students, and university support personnel Marchand and Gutierrez (2012)noted the importance of emotional student support in the graduate online environment. The utility of value, or how valuable the tasks were in the students’ personal and professional goals, along with relevance of instruction and academic self-efficacy, interacted with both positive and negative emotions, ranging from hope to anxiety, respectively. The findings in Hwu’s (2011)and Ice (2006)study also suggested the importance of implementing additional administrative support, proper recognition of achievements and investing in continuous professional development for instructors teaching graduate level online courses. Multiple studies including Evans (2013), Kahrhoff (2005), Pattison (2017)and Holland (2006)also noted the importance of adequate student socialization, the use of student peer mentors, the use of allusions to the group or classroom, and supplemental online support for students. Student socialization could include students interacting with other students through online discussion portals, group messaging, or other ways of being able to communicate with peers. This was particularly important for first-year graduate students in order to improve their chances for retention and the overall success in online courses. According to research done with doctoral students of an education technology program at a Florida University, Kumar (2014)conceptualized that in order to decrease student anxiety and increase satisfaction during e-learning, instructors need to have consistent communication, structure and timelines to yield better satisfaction. Maryannakis (2009)studied graduate students over one academic term to determine the best graduate online course design for a research methods course, finding that the structured “comprehensive objectivist” design was significantly more effective than its unstructured counterpart on graduate learners’ competence in research methods. Merena (2006)found that student interaction with the instructor, quality of the course materials, improved evaluation methods, delivery methods and discipline all factored into a student’s satisfaction and success in the online distance education environment. Similarly, Keeler’s (2006)quantitative approach to determine the relationship between types of interaction and student satisfaction in master’s level distant education showed learner-instructor interaction contributed the most to overall student satisfaction in distant education. Lastly, Ruiz (2007)analyzed overall student satisfaction in online education students and conventional-campus-based students from the same graduate education program and found that, distant education students were more satisfied than their counter-parts enrolled in the conventional campus-based programs.
6.1. Practical Significance of the Findings on Students
As graduate-level online students, we know how essential it is for instructors to practice best pedagogy so that students can achieve the greatest educational outcomes. Understanding which pedagogical practices are best suited for graduate-level online students is essential for positive learning outcomes. There needs to be solid teaching practices so that even at-risk students can be successful in their courses. We discovered that the best pedagogical practice is for instructors to have good interaction with their learners, including good communication and relations. Groeling (2004)found that the interaction between instructors and students is the “mechanism for student learning” (p. 188). Having good interaction with students can help instructors know whether each student is learning to the best of their ability. Hegngi (1997)also emphasized that instructors need to improve on their interactions with students in order for them not feel isolated. If instructors do not interact with their students, then the latter will feel isolated and unmotivated, resulting in lower educational outcomes. Good interaction can help instructors improve in teaching and students in learning. Table 1outlines the significance and impact these findings have on students.
6.2. Practical Significance of the Findings on Faculty
Multiple studies showed better educational outcomes in online setting when instructors adapted their pedagogy to the students. Therefore, one of the practical implications of this systematic review for faculty would be helping students become successful online learners by adapting parts of the course based on a pre-course evaluation of the students’ distance education experience, preferred method of learning and computer competency. A short automated survey could be implemented and administered to students immediately after registration for the course allowing, enough time for faculty to review the results before beginning of the semester. An emphasis should be placed on the faculty development and training. Multiple studies showed that instructors need more technical support, as discussed by Wang (2007), more administrative support, proper recognition of their achievements, professional development, and an enhanced role for centralized support for faculty engaged in online learning. University administrators should consider implementing 24-hour online support services available to the online students in order to improve their chances at succeeding in online courses. This way students can voice their concerns, ask additional questions or clear any misunderstandings associated with their online learning courses. Table 1summarizes the importance of these findings for faculty.
6.3. Practical Significance of the Findings on Institutions of Higher Education (IHE)
Today’s reality is: online learning is here to stay; it is neither a fad nor a trend(Betts, 2017: p. 2). Moreover, Betts (2017)explained how the ubiquity of technology has made taking online courses the new norm. Students who seek flexible course formats largely drive enrollment for online courses at IHE as well as degree programs that support career placement and that readily adjust to students’ personal and professional schedules(Betts, 2017). Our review findings highlighted the importance for all IHE to adopt, evaluate and continuously develop web-based innovative curricula within their existing academic frameworks. This is particularly demonstrated in Allen and Seaman’s (2014)study which discussed academic leaders’ optimism about the continued growth of online learning and its significant impact on higher education students. Interestingly, smaller universities were most passionate about how the improvement of technology helps universities decrease the costs for development and delivery of online courses, when compared to face-to-face courses(Allen & Seaman, 2014). A vast amount of larger universities also agreed with the concept, yet it is important to note most are large, public universities, where costs per course are low, compared to smaller, private institutions. Merena (2006)emphasized the need for institutions of higher education to refine their institutional missions, goals and audience to gear them toward e-learning. It was also expressed that fostering students’ engagement and acclimation early in on-line course studies was essential for graduate-level online learning. Universities must focus on implementing an ongoing quality
Table 1. Practical significance of the findings on faculty, students, and institutions.
6.4. Limitations and Implications for Future Studies
Limitations of this narrative descriptive systematic review included:
・ Given that e-learning is a relatively new topic, paired with rigid review guidelines, this systematic review was limited to 24 included literature sources.
・ Our primary studies did not include other systematic reviews in order to help avoid double counting studies included in both systematic reviews and to reduce the likelihood of forming inferences and conclusions biased by the results of reviews done by others.
・ We reported the findings in the context of an ensemble of recommendations related to best pedagogical practices for online graduate learning considering the heterogeneous nature of graduate courses/programs examined.
・ Though majority of included studies evaluated graduate students, a few consisted of a mix of graduate and undergraduate students; for the purpose of this systematic review only study results from graduate students were considered. In addition, seven mixed method studies were included in this systematic review, but only quantitative results were considered.
・ Only studies where researchers explored the use of pedagogical practices in graduate-level online learning were included in this systematic review.
・ There was a lack of randomized controlled studies in this systematic review therefore higher quality studies are needed to make a better conclusion.
・ This descriptive systematic review concludes with results based on only 24 included literature sources and their implications for graduate online learning community, along with recommendations for future research.
Online Learning Consortium (2016)stated “the number of students taking online courses grew to 5.8 million nationally, continuing a growth trend that has been consistent for 13 years.” It is projected that each year more and more graduate students will be enrolling in online courses. According to the most recent study done by Babson Survey Research Group, “the number of higher education students taking at least one distance education course in 2015 is up 3.9% over the previous year”(Online Learning Consortium, 2016). Despite this growth, “some faculty members have become less confident in these online programs”(Online Learning Consortium, 2016). This suggests that there is a greater need for more quantitative studies to be conducted on best pedagogical practices specifically in graduate-level online learning for institutions to have a better understanding on which pedagogical practices are most beneficial for student’s educational outcomes.
As graduate-level online students we chose a subject that has been understudied, and therefore is important for online institutions, students and faculty to understand. After conducting our systematic review, we found that the best pedagogical practices associated with achieving the greatest educational outcomes for graduate-level online students were having positive academic discipline and attitude towards online education, and clear, timely and consistent teacher-student communication. We also discovered that the most effective pedagogical practice used by online instructors was student-teacher interaction and communication. It is important for online instructors to go the extra mile to establish effective communication with students in order to ensure the students understand the course content and learning to the best of their ability. Instructors can establish effective communication by reaching out to each student periodically to check on their progress in the course. The second best practice suggested was for online instructors to evaluate each student’s distance education experience, preferred method of learning and computer competency in the beginning of the course. Instructors can do this by conducting a survey and incorporating the results into their pedagogical practices in online setting. This will help instructors develop better pedagogical practices in their online courses, which will help students achieve the best educational outcome and timely graduation.
Worldwide we are in the Digital Age, where traditional industries have shifted to a period where our economies are based on information technology and computerization. In turn, this era demands expeditious processing of information, supply of resources and delivery of results. Most notably, education and healthcare are two key industries whose existence and advancement depend heavily on technology. Given the continuous growth of distant education on the graduate level, the identification and adoption of the best pedagogical practices in e-learning will benefit stakeholders who have major investments in online education. These stakeholders include faculty and students. Additionally, they include academic institutions in the health, education and technology industries, and the systematic review process offers searches that are replicable, highly trustworthy and reliable, proving quintessential for those sectors.
Conflict of Interest
The authors certify we had no commercial or financial involvements, funding or benefits from industry or elsewhere to conduct this systematic review.
This research did not receive any specific grant from any funding agency in the public, commercial, or not-for-profit sectors.
Appendix A: Best Graduate Pedagogical Practices Summary Table
Researchers: Gleyzer, Javed, Pardino
Appendix B: Evidence Table
 Allen, I. E., & Seaman, J. (2014). Grade Change: Tracking Online Education in the United States. Babson Survey Research Group and Quahog Research Group, LLC.
 Betts, K. (2017). The Growth of Online Learning: How Universities Must Adjust to the New Norm. Philadelphia, PA: Drexel University.
 Evans, N. S. (2013). A Cross-Sectional Descriptive Study of Graduate Students’ Perceptions of Learning Effectiveness in Face-to-Face and Online Courses. PhD Dissertation, New Castle, DE: Wilmington University.
 Groeling, J. P. (2004). Does Technology Make a Difference in the Classroom? A Comparison of Online and Traditional Courses Employing Models of Competence and Uses and Gratifications. PhD Dissertation, Lexington, KY: University of Kentucky.
 Hamrick, S. M. (2004). A Theory and Behavior-Based Analysis on the Nature of Situational and Interactive Influences on Online Students’ Perceptions of Motivation. PhD Dissertation, Ann Arbor, MI: Regent University.
 Hegngi, Y. N. (1997). Online Teaching and Learning: A Description of the Development of the Media Technology and Diversity Online Course and Its Electronic Discourse Analysis. PhD Dissertation, Ann Arbor, MI: Virginia Polytechnic Institute & State University.
 Ice, P. R. (2006). The Relationship between Technical Support and Pedagogical Guidance Provided to Faculty and Student Satisfaction in Online Courses. PhD Dissertation, Ann Arbor, MI: West Virginia University.
 Kahrhoff, J. B. (2005). Exploring the Relationship between Adult Development and Support Service Needs of Online Graduate Students. PhD Dissertation, Ann Arbor, MI: The University of Missouri-St. Louis.
 Kovanovic, V., Gasevic, D., Joksimovic, S., Hatala, M., & Adesope, O. (2015). Analytics of Communities of Inquiry: Effects of Learning Technology Use on Cognitive Presence in Asynchronous Online Discussions. Internet and Higher Education, 27, 74-89.
 Kumar, S. (2014). Signature Pedagogy, Implementation and Evaluation of an Online Program That Impacts Educational Practice. Internet and Higher Education, 21, 60-67.
 Kuna, A. S. (2012). Learner Interaction Patterns and Student Perceptions toward Using Selected Tools in an Online Course Management System. PhD Dissertation, Ann Arbor, MI: Iowa State University.
 Levinson, C. C. (2012). Factors That Influence the Ability of Preservice Teachers to Apply English Language Arts Pedagogy in Guided Practice. PhD Dissertation, Ann Arbor, MI: University of Southern California.
 Marchand, G. C., & Gutierrez, A. P. (2012). The Role of Emotion in the Learning Process: Comparisons between Online and Face-to-Face Learning Settings. Internet and Higher Education, 15, 150-160.
 Mozzani-Miller, P. (2006). A Comparison of Learning Experienced by Students Who Work On-Line Versus Students Who Work Off-line in Distance Education Graduate Courses: A Mixed Method Study. PhD Dissertation, Lincoln, NE: University of Nebraska.
 Online Learning Consortium (2016). Babson Study: Distance Education Enrollment Growth Continues.
 Pattison, A. B. (2017). An Exploratory Study of the Relationship between Faculty Social Presence and Online Graduate Student Achievement, Satisfaction, and Persistence. PhD Dissertation, Ann Arbor, MI: Grand Canyon University.
 Randle, D. E. (2013). An Analysis of Interactions and Outcomes Associated with an Online Professional Development Course for Science Teachers. PhD Dissertation, Ann Arbor, MI: Columbia University.
 Ruhlandt, R. R. (2010). Differences in Retention, Social Presence, Cognitive Presence, and Teaching Presence in Fully Online and Blended Courses. PhD Dissertation, Ann Arbor, MI: Regent University.