Interactive Graphics for Presentation and Exploration of Student Performance Profiles in IGLU/TIMSS 2011 Educational Surveys
Affiliation(s)
TUM School of Education and Centre for International Student Assessment, TU Munich, Munich, Germany.
TUM School of Education and Centre for International Student Assessment, TU Munich, Munich, Germany.
ABSTRACT
This paper advocates the application of interactive graphics as a qualitative research method for comparative large-scale assessments, known to supplement and extend analytical techniques. Graphics can be easily generated for effective display of information. Interaction is essential for exploring data in a flexible and controlled manner. The national results of the common sample of the large-scale educational assessment studies Internationale Grundschul-Lese-Untersuchung (IGLU)/ Trends in International Mathematics and Science Study (TIMSS) 2011 demonstrated the effectiveness of interactive graphics. The performance profiles of elementary school students in terms of the competency domains of reading, mathematics, and science are analyzed. Furthermore, spineplots are used to investigate the background characteristics of students of different performance types in order to identify possible educational disadvantage. The study aims to simplify data exploration before reporting, and to more effectively present the findings obtained from comparison studies on school achievement.
This paper advocates the application of interactive graphics as a qualitative research method for comparative large-scale assessments, known to supplement and extend analytical techniques. Graphics can be easily generated for effective display of information. Interaction is essential for exploring data in a flexible and controlled manner. The national results of the common sample of the large-scale educational assessment studies Internationale Grundschul-Lese-Untersuchung (IGLU)/ Trends in International Mathematics and Science Study (TIMSS) 2011 demonstrated the effectiveness of interactive graphics. The performance profiles of elementary school students in terms of the competency domains of reading, mathematics, and science are analyzed. Furthermore, spineplots are used to investigate the background characteristics of students of different performance types in order to identify possible educational disadvantage. The study aims to simplify data exploration before reporting, and to more effectively present the findings obtained from comparison studies on school achievement.
KEYWORDS
Data Exploration, Interactive Graphics, Large-Scale Assessment, IGLU/TIMSS 2011, Mondrian, R, Performance Profiles, Background Characteristics, Educational Disadvantage
Data Exploration, Interactive Graphics, Large-Scale Assessment, IGLU/TIMSS 2011, Mondrian, R, Performance Profiles, Background Characteristics, Educational Disadvantage
Cite this paper
Ünlü, A. and Gschrey, B. (2015) Interactive Graphics for Presentation and Exploration of Student Performance Profiles in IGLU/TIMSS 2011 Educational Surveys. Open Journal of Social Sciences, 3, 122-136. doi: 10.4236/jss.2015.39018.
Ünlü, A. and Gschrey, B. (2015) Interactive Graphics for Presentation and Exploration of Student Performance Profiles in IGLU/TIMSS 2011 Educational Surveys. Open Journal of Social Sciences, 3, 122-136. doi: 10.4236/jss.2015.39018.
References
[1] OECD (2012) PISA 2009 Technical Report. OECD Publishing, Paris. [2] Martin, M.O., Mullis, I.V.S. and Kennedy, A.M. (2007) PIRLS 2006 Technical Report. TIMSS & PIRLS International Study Center, Chestnut Hill. [3] Martin, M.O. and Mullis, I.V.S. (2013) Methods and Procedures in TIMSS and PIRLS 2011. TIMSS & PIRLS International Study Center, Chestnut Hill. [4] Mullis, I.V.S., Martin, M.O., Kennedy, A.M., Trong, T.L. and Sainsbury, M. (2009) PIRLS 2011 Assessment Framework. TIMSS & PIRLS International Study Center, Chestnut Hill. [5] Bos, W., Tarelli, I., Bremerich-Vos, A. and Schwippert, K. (2012) IGLU 2011: Lesekompetenzen von Grundschulkindern in Deutschland im Internationalen Vergleich. Waxmann, Münster. [6] Bos, W., Wendt, H., Köller, O. and Selter, C. (2012) TIMSS 2011: Mathematische und Naturwissenschaftliche Kompetenzen von Grundschulkindern in Deutschland im Internationalen Vergleich. Waxmann, Münster. [7] Mullis, I.V.S., Martin, M.O., Ruddock, G.J., O’Sullivan, C.Y. and Preuschoff, C. (2009) TIMSS 2011 Assessment Frameworks. TIMSS & PIRLS International Study Center, Chestnut Hill. [8] Unwin, A.R., Chen, C. and Härdle, W. (2007) Computational Statistics and Data Visualization. Sonderforschungsbereich 649: Ökonomisches Risiko, Diskussionspapiere. Humboldt-Universität, Berlin. [9] Theus, M. (2002) Interactive Data Visualization Using Mondrian. Journal of Statistical Software, 7, 11-22. [10] ünlü, A. and Sargin, A. (2009) Interactive Visualization of Assessment Data: The Software Package Mondrian. Applied Psychological Measurement, 33, 148-156.
http://dx.doi.org/10.1177/0146621608319511 [11] ünlü, A. and Malik, W. (2011) Interactive Glyph Graphics of Multivariate Data in Psychometrics: The Software Package Gauguin. Methodology: European Journal of Research Methods for the Behavioral and Social Sciences, 7, 134-144. http://dx.doi.org/10.1027/1614-2241/a000031 [12] Theus, M. and Urbanek, S. (2009) Interactive Graphics for Data Analysis: Principles and Examples. CRC Press, Boca Raton. [13] R Core Team (2014) R: A Language and Environment for Statistical Computing. R Foundation for Statistical Com- puting, Vienna. [14] Chen, C., Härdle, W. and Unwin, A. (2008) Handbook of Data Visualization. Springer, Berlin, Heidelberg. [15] Adams, R.J., Wilson, M.R. and Wang, W.L. (1997) The Multidimensional Random Coefficients Multinomial Logit Model. Applied Psychological Measurement, 21, 1-24.
http://dx.doi.org/10.1177/0146621697211001 [16] Bacher, J. and Vermunt, J.K. (2010) Analyse Latenter Klassen. In: Wolf, C. and Best, H., Eds., Handbuch der Sozialwissenschaftlichen Datenanalyse, VS-Verlag für Sozialwissenschaften, Wiesbaden, 553-574. http://dx.doi.org/10.1007/978-3-531-92038-2_22 [17] Lazarsfeld, P.F. and Henry, N.W. (1968) Latent Structure Analysis. Houghton Mifflin, New York. [18] Bos, W., Wendt, H., ünlü, A., Valtin, R., Euen, B., Kasper, D. and Tarelli, I. (2012) Leistungsprofile von Viertklässlerinnen und Viertklässlern in Deutschland. In: Bos, W., Tarelli, I., Bremerich-Vos, A. and Schwippert, K., Eds., IGLU 2011: Lesekompetenzen von Grundschulkindern in Deutschland im Internationalen Vergleich, Waxmann, Münster, 226-257. [19] Burnham, K.P. and Anderson, D.R. (2002) Model Selection and Multi-Model Inference: A Practical Information- Theoretic Approach. Springer, New York. [20] Eshghi, A., Haughton, D., Legrand, P., Skaletsky, M. and Woolford, S. (2011) Identifying Groups: A Comparison of Methodologies. Journal of Data Science, 9, 271-291. [21] Baumert, J., Stanat, P. and Watermann, R. (2006) Herkunftsbedingte Disparitäten im Bildungswesen: Differenzielle Bildungsprozesse und Probleme der Verteilungsgerechtigkeit. VS-Verlag für Sozialwissenschaften, Wiesbaden. [22] Baur, C. (2011) Bildungsbenachteiligung von Kindern und Jugendlichen mit Migrationshintergrund durch Soziale und Ethnische Segregation. In: Frey, R. and Koch, F., Eds., Gesellschaft, Governance, Gestaltung, LIT, Wien, 91-108. [23] Kuhlmann, C. (2011) Bildungsbenachteiligung in der Europäischen Diskussion—Angrenzungen für Strategien Gegen Soziale Ausgrenzungen. In: Benz, B., Boeckh, J. and Mogge-Grotjahn, H., Eds., Soziale Politik—Soziale Lage— Soziale Arbeit, VS-Verlag für Sozialwissenschaften, Wiesbaden, 282-298. [24] Gold, A. (2011) Lernschwierigkeiten. Ursachen, Diagnostik, Intervention. Kohlhammer, Stuttgart. [25] Leitner, W.G., Ortner, A. and Ortner, R. (2008) Handbuch Verhaltens- und Lernschwierigkeiten. Beltz, Weinheim, Basel. [26] Schümer, G. (2004) Zur Doppelten Benachteiligung von Schülern aus Unterprivilegierten Gesellschaftsschichten im Deutschen Schulwesen. In: Schümer, G., Tillmann, K.-J. and Weiß, M., Eds., Die Institution Schule und die Lebenswelt der Schüler: Vertiefende Analysen der PISA-2000-Daten zum Kontext von Schülerleistungen, VS-Verlag für Sozialwissenschaften, Wiesbaden, 73-114. http://dx.doi.org/10.1007/978-3-531-90348-4_4 [27] Hummel, J. (1996) Linked Bar Charts: Analysing Categorical Data Graphically. Computational Statistics, 11, 23-33. [28] Graham, J.W. (2009) Missing Data Analysis: Making It Work in the Real World. Annual Review of Psychology, 60, 549-579. http://dx.doi.org/10.1146/annurev.psych.58.110405.085530 [29] Kasper, D., ünlü A. and Gschrey, B. (2014) Sensitivity Analyses for the Mixed Coefficients Multinomial Logit Model. In: Spiliopoulou, M., Schmidt-Thieme, L. and Janning, R., Eds., Data Analysis, Machine Learning and Knowledge Discovery, Springer, Heidelberg, 389-396. http://dx.doi.org/10.1007/978-3-319-01595-8_42 [30] Inselberg, A. (2009) Parallel Coordinates: Visual Multidimensional Geometry and Its Applications. Springer, New York. http://dx.doi.org/10.1007/978-0-387-68628-8 [31] Unwin, A.R., Volinsky, C. and Winkler, S. (2003) Parallel Coordinates for Exploratory Modelling Analysis. Computational Statistics & Data Analysis, 43, 553-564. http://dx.doi.org/10.1016/S0167-9473(02)00292-X [32] Wegman, E.J. (1990) Hyperdimensional Data Analysis Using Parallel Coordinates. Journal of the American Statistical Association, 85, 664-675. http://dx.doi.org/10.1080/01621459.1990.10474926 [33] Arnold, K. (Ed.) (2008) Handbuch Förderung: Grundlagen, Bereiche und Methoden der Individuellen Förderung von Schülern. Beltz, Weinheim, Basel. [34] Sarkar, D. (2008) Lattice: Multivariate Data Visualization with R. Springer, New York. http://dx.doi.org/10.1007/978-0-387-75969-2 [35] Cleveland, W.S. and Becker, R.A. (1996) S-PLUS Trellis Graphics User’s Manual. MathSoft, Inc., Seattle. [36] Robbins, N.B. (2005) Creating More Effective Graphs. John Wiley & Sons, Inc., Hoboken. [37] Foscht, T., Angerer, T. and Swoboda, B. (2007) Mixed Methods. In: Renate, B. and Holzmüller, H., Eds., Qualitative Marktforschung: Konzepte, Methoden, Analysen, Gabler, Wiesbaden, 247-259. http://dx.doi.org/10.1007/978-3-8349-9258-1_16 [38] Johnson, R.B., Onwuegbuzie, A.J. and Turner, L.A. (2007) Toward a Definition of Mixed Methods Research. Journal of Mixed Methods Research, 1, 112-133. http://dx.doi.org/10.1177/1558689806298224 [39] Onwuegbuzie, A.J. and Dickinson, W.B. (2008) Mixed Methods Analysis and Information Visualization: Graphical Display for Effective Communication of Research Results. The Qualitative Report, 13, 204-225.
[1] OECD (2012) PISA 2009 Technical Report. OECD Publishing, Paris. [2] Martin, M.O., Mullis, I.V.S. and Kennedy, A.M. (2007) PIRLS 2006 Technical Report. TIMSS & PIRLS International Study Center, Chestnut Hill. [3] Martin, M.O. and Mullis, I.V.S. (2013) Methods and Procedures in TIMSS and PIRLS 2011. TIMSS & PIRLS International Study Center, Chestnut Hill. [4] Mullis, I.V.S., Martin, M.O., Kennedy, A.M., Trong, T.L. and Sainsbury, M. (2009) PIRLS 2011 Assessment Framework. TIMSS & PIRLS International Study Center, Chestnut Hill. [5] Bos, W., Tarelli, I., Bremerich-Vos, A. and Schwippert, K. (2012) IGLU 2011: Lesekompetenzen von Grundschulkindern in Deutschland im Internationalen Vergleich. Waxmann, Münster. [6] Bos, W., Wendt, H., Köller, O. and Selter, C. (2012) TIMSS 2011: Mathematische und Naturwissenschaftliche Kompetenzen von Grundschulkindern in Deutschland im Internationalen Vergleich. Waxmann, Münster. [7] Mullis, I.V.S., Martin, M.O., Ruddock, G.J., O’Sullivan, C.Y. and Preuschoff, C. (2009) TIMSS 2011 Assessment Frameworks. TIMSS & PIRLS International Study Center, Chestnut Hill. [8] Unwin, A.R., Chen, C. and Härdle, W. (2007) Computational Statistics and Data Visualization. Sonderforschungsbereich 649: Ökonomisches Risiko, Diskussionspapiere. Humboldt-Universität, Berlin. [9] Theus, M. (2002) Interactive Data Visualization Using Mondrian. Journal of Statistical Software, 7, 11-22. [10] ünlü, A. and Sargin, A. (2009) Interactive Visualization of Assessment Data: The Software Package Mondrian. Applied Psychological Measurement, 33, 148-156.
http://dx.doi.org/10.1177/0146621608319511 [11] ünlü, A. and Malik, W. (2011) Interactive Glyph Graphics of Multivariate Data in Psychometrics: The Software Package Gauguin. Methodology: European Journal of Research Methods for the Behavioral and Social Sciences, 7, 134-144. http://dx.doi.org/10.1027/1614-2241/a000031 [12] Theus, M. and Urbanek, S. (2009) Interactive Graphics for Data Analysis: Principles and Examples. CRC Press, Boca Raton. [13] R Core Team (2014) R: A Language and Environment for Statistical Computing. R Foundation for Statistical Com- puting, Vienna. [14] Chen, C., Härdle, W. and Unwin, A. (2008) Handbook of Data Visualization. Springer, Berlin, Heidelberg. [15] Adams, R.J., Wilson, M.R. and Wang, W.L. (1997) The Multidimensional Random Coefficients Multinomial Logit Model. Applied Psychological Measurement, 21, 1-24.
http://dx.doi.org/10.1177/0146621697211001 [16] Bacher, J. and Vermunt, J.K. (2010) Analyse Latenter Klassen. In: Wolf, C. and Best, H., Eds., Handbuch der Sozialwissenschaftlichen Datenanalyse, VS-Verlag für Sozialwissenschaften, Wiesbaden, 553-574. http://dx.doi.org/10.1007/978-3-531-92038-2_22 [17] Lazarsfeld, P.F. and Henry, N.W. (1968) Latent Structure Analysis. Houghton Mifflin, New York. [18] Bos, W., Wendt, H., ünlü, A., Valtin, R., Euen, B., Kasper, D. and Tarelli, I. (2012) Leistungsprofile von Viertklässlerinnen und Viertklässlern in Deutschland. In: Bos, W., Tarelli, I., Bremerich-Vos, A. and Schwippert, K., Eds., IGLU 2011: Lesekompetenzen von Grundschulkindern in Deutschland im Internationalen Vergleich, Waxmann, Münster, 226-257. [19] Burnham, K.P. and Anderson, D.R. (2002) Model Selection and Multi-Model Inference: A Practical Information- Theoretic Approach. Springer, New York. [20] Eshghi, A., Haughton, D., Legrand, P., Skaletsky, M. and Woolford, S. (2011) Identifying Groups: A Comparison of Methodologies. Journal of Data Science, 9, 271-291. [21] Baumert, J., Stanat, P. and Watermann, R. (2006) Herkunftsbedingte Disparitäten im Bildungswesen: Differenzielle Bildungsprozesse und Probleme der Verteilungsgerechtigkeit. VS-Verlag für Sozialwissenschaften, Wiesbaden. [22] Baur, C. (2011) Bildungsbenachteiligung von Kindern und Jugendlichen mit Migrationshintergrund durch Soziale und Ethnische Segregation. In: Frey, R. and Koch, F., Eds., Gesellschaft, Governance, Gestaltung, LIT, Wien, 91-108. [23] Kuhlmann, C. (2011) Bildungsbenachteiligung in der Europäischen Diskussion—Angrenzungen für Strategien Gegen Soziale Ausgrenzungen. In: Benz, B., Boeckh, J. and Mogge-Grotjahn, H., Eds., Soziale Politik—Soziale Lage— Soziale Arbeit, VS-Verlag für Sozialwissenschaften, Wiesbaden, 282-298. [24] Gold, A. (2011) Lernschwierigkeiten. Ursachen, Diagnostik, Intervention. Kohlhammer, Stuttgart. [25] Leitner, W.G., Ortner, A. and Ortner, R. (2008) Handbuch Verhaltens- und Lernschwierigkeiten. Beltz, Weinheim, Basel. [26] Schümer, G. (2004) Zur Doppelten Benachteiligung von Schülern aus Unterprivilegierten Gesellschaftsschichten im Deutschen Schulwesen. In: Schümer, G., Tillmann, K.-J. and Weiß, M., Eds., Die Institution Schule und die Lebenswelt der Schüler: Vertiefende Analysen der PISA-2000-Daten zum Kontext von Schülerleistungen, VS-Verlag für Sozialwissenschaften, Wiesbaden, 73-114. http://dx.doi.org/10.1007/978-3-531-90348-4_4 [27] Hummel, J. (1996) Linked Bar Charts: Analysing Categorical Data Graphically. Computational Statistics, 11, 23-33. [28] Graham, J.W. (2009) Missing Data Analysis: Making It Work in the Real World. Annual Review of Psychology, 60, 549-579. http://dx.doi.org/10.1146/annurev.psych.58.110405.085530 [29] Kasper, D., ünlü A. and Gschrey, B. (2014) Sensitivity Analyses for the Mixed Coefficients Multinomial Logit Model. In: Spiliopoulou, M., Schmidt-Thieme, L. and Janning, R., Eds., Data Analysis, Machine Learning and Knowledge Discovery, Springer, Heidelberg, 389-396. http://dx.doi.org/10.1007/978-3-319-01595-8_42 [30] Inselberg, A. (2009) Parallel Coordinates: Visual Multidimensional Geometry and Its Applications. Springer, New York. http://dx.doi.org/10.1007/978-0-387-68628-8 [31] Unwin, A.R., Volinsky, C. and Winkler, S. (2003) Parallel Coordinates for Exploratory Modelling Analysis. Computational Statistics & Data Analysis, 43, 553-564. http://dx.doi.org/10.1016/S0167-9473(02)00292-X [32] Wegman, E.J. (1990) Hyperdimensional Data Analysis Using Parallel Coordinates. Journal of the American Statistical Association, 85, 664-675. http://dx.doi.org/10.1080/01621459.1990.10474926 [33] Arnold, K. (Ed.) (2008) Handbuch Förderung: Grundlagen, Bereiche und Methoden der Individuellen Förderung von Schülern. Beltz, Weinheim, Basel. [34] Sarkar, D. (2008) Lattice: Multivariate Data Visualization with R. Springer, New York. http://dx.doi.org/10.1007/978-0-387-75969-2 [35] Cleveland, W.S. and Becker, R.A. (1996) S-PLUS Trellis Graphics User’s Manual. MathSoft, Inc., Seattle. [36] Robbins, N.B. (2005) Creating More Effective Graphs. John Wiley & Sons, Inc., Hoboken. [37] Foscht, T., Angerer, T. and Swoboda, B. (2007) Mixed Methods. In: Renate, B. and Holzmüller, H., Eds., Qualitative Marktforschung: Konzepte, Methoden, Analysen, Gabler, Wiesbaden, 247-259. http://dx.doi.org/10.1007/978-3-8349-9258-1_16 [38] Johnson, R.B., Onwuegbuzie, A.J. and Turner, L.A. (2007) Toward a Definition of Mixed Methods Research. Journal of Mixed Methods Research, 1, 112-133. http://dx.doi.org/10.1177/1558689806298224 [39] Onwuegbuzie, A.J. and Dickinson, W.B. (2008) Mixed Methods Analysis and Information Visualization: Graphical Display for Effective Communication of Research Results. The Qualitative Report, 13, 204-225.