Norwich Medical School, Faculty of Medicine and Health Sciences, University of East Anglia, Norwich, UK.
Background: There is a need to evaluate perceptions of the educational environment of training institutions for health professionals as part of any assessment of quality standards for education. The Dundee Ready Education Environment Measure (DREEM) is a widely used tool for evaluating the educational environment of medical and other health schools. However, methods of analysis reported in the published DREEM literature are inconsistent which could lead to misinterpretation of areas for change and, additionally, this makes comparison between institutions difficult. Those involved in course evaluation are usually not statisticians and there are no guidelines on DREEM’s reporting or statistical analysis. This paper aims to clarify the choice of methods for the analysis of the DREEM. Method: The statistical literature, typical properties of DREEM data and the results from a series of statistical simulations were used to inform our recommendations. Results: We provide a set of guidelines for the analysis and reporting of the DREEM. In particular, we provide evidence that when comparing independent samples of Likert response data similar to that generated by the DREEM, the non-parametric Wilcoxon Mann Whitney test performs well. Further, one should be wary of using non-parametric methods on matched samples of such data as they may be overly ready to reject null hypothesis. Conclusions: Our recommendations have the potential to improve the accuracy and consistency with which the inadequacies in the medical school environment can be identified and assess the success of any changes. They should also facilitate comparison between different institutions using the DREEM.
DREEM; Likert; Educational Environment; Evaluation; Medical Education; Simulation; Statistical Test
Swift, L. , Miles, S. & Leinster, S. (2013). The Analysis and Reporting of the Dundee Ready Education Environment Measure (DREEM): Some Informed Guidelines for Evaluators. Creative Education, 4, 340-347. doi: 10.4236/ce.2013.45050.
[1] Agresti, A. (2002). Categorical data analysis. 2nd Edition, Hoboken, NJ: Wiley. doi:10.1002/0471249688 [2] Blair, R. C. (1981). A reaction to “Consequences of failure to meet assumptions underlying the fixed effects analysis of variance and covariance”. Review of Educational Research, 51, 499-507. doi:10.3102/00346543051004499 [3] Carifio, J., & Perla, R. J. (2007). Ten common misunderstandings, misconceptions, persistent myths and urban legends about Likert scales and Likert response formats and their antidotes. Journal of Social Sciences, 3, 106-116. [4] Carifio, J., & Perla, R. J. (2008). Resolving the 50-year debate around using and misusing Likert scales. Medical Education, 42, 1150-1152. doi:10.1111/j.1365-2923.2008.03172.x [5] Feise, R. J. (2002). Do multiple outcome measures require p-value ad justment? BMC Medical Research Methodology, 2, 8. doi:10.1186/1471-2288-2-8 [6] Genn, J. M. (2001a). AMEE Medical Education Guide No. 23 (Part 1). Curriculum, environment, climate, quality and change in medical education: A unifying perspective. Medical Teacher, 23, 337-344. doi:10.1080/01421590120063330 [7] Genn, J. M. (2001b). AMEE Medical Education Guide No. 23 (Part 2). Curriculum, environment, climate, quality and change in medical education: A unifying perspective. Medical Teacher, 23, 445-454. [8] Glass, G. V., Peckham, P. D., & Sanders, J. R. (1972). Consequences of failure to meet assumptions underlying the fixed effects analysis of variance and covariance. Review of Educational Research, 42, 237-288. doi:10.3102/00346543042003237 [9] Jamieson, S. (2004). Likert scales: How to (ab)use them. Medical Education, 38, 1212-1218. doi:10.1111/j.1365-2929.2004.02012.x [10] McAleer, S., & Roff, S. (2001). A practical guide to using the Dundee Ready Education Environment Measure (DREEM). In J. M. Genn (Ed.), Curriculum, environment, climate, quality and change in medical education: A unifying perspective (pp. 29-33). AMEE Education Guide no. 23. Scotland: AMEE. [11] Miles, S., & Leinster, S. J. (2007). Medical students’ perceptions of their educational environment: Expected versus actual perceptions. Medical Education, 41, 265-272. doi:10.1111/j.1365-2929.2007.02686.x [12] Miles, S., & Leinster, S. J. (2009). Comparing staff and student perceptions of the student experience at a new medical school. Medical Teacher, 31, 539-546. doi:10.1080/01421590802139732 [13] Miles, S., Swift, L., & Leinster, S. J. (2012). The Dundee Ready Education Environment Measure (DREEM): A review of its adoption and use. Medical Teacher, 34, e620-e634. doi:10.3109/0142159X.2012.668625 [14] Norman, G. (2010). Likert scales, levels of measurement and the “laws” of statistics. Advances in Health Science Education, 15, 625-632. doi:10.1007/s10459-010-9222-y [15] Pell, G. (2005). Use and misuse of Likert scales. Medical Education, 3, 970. doi:10.1111/j.1365-2929.2005.02237.x [16] Roff, S., McAleer, S., Harden, R. M., Al-Qahtani, M., Ahmed, A. U., Deza, H., Groenen, G., & Primparyon, P. (1997). The Development and validation of the Dundee Ready Education Environment Measure (DREEM). Medical Teacher, 19, 295-299. doi:10.3109/01421599709034208 [17] Srivastava, A. B. L. (1959). Effect of non-normality on the power of the analysis of variance test. Biometrika, 4, 114-122. [18] Till, H. (2004). Identifying the perceived weaknesses of a new curriculum by means of the Dundee Ready Education Environment Measure (DREEM). Medical Teacher, 26, 39-45. doi:10.1080/01421590310001642948 [19] Till, H. (2005). Climate studies: Can students’ perception of the ideal educational environment be of used for institutional planning and re source utilization? Medical Teacher, 27, 332-337. doi:10.1080/01421590400029723