In order to equip younger generations with 21st century skills, developing students’ higher order thinking skills has been emphasized in educational objectives throughout the world ( Binkley et al., 2012 ; Greiff, Niepel, & Wustenberg, 2015 ; Gut, 2011 ). Since the Thinking Skills Movement in the UK and the Critical Thinking Movement in the USA in 1970s ( Paul, 1997 ; Resch, 2008 ), a variety of thinking skills interventions have emerged ( Adey, 1988 ; Buzan & Buzan, 1996 ; Feuerstein & Jensen, 1980 ; Hyerle, 2011 ; Lipman, 1976 ; McGuinness, Eakin, Curry, & Sheehy, 2007 ; Novak, 1990 ). In defense of their thinking programs, extensive empirical studies have been carried out to examine their respective effects ( Adey, Robertson, & Venville, 2002 ; Dewey & Bento, 2009 ; Mashal & Kasirer, 2011 ; Mbano, 2003 ; Oliver, Venville, & Adey, 2012 ; Sunseri, 2011 ; Tripp, 1980 ). However, measures or scales used in most studies were designed specifically for their particular interventions and these purpose-designed measures always face challenges on their objectivities and universalities ( Burke & Williams, 2012 ). Furthermore, due to the varieties in participants, it remains problematic to compare the effects of various thinking interventions in general ( Burke & Williams, 2008 ).
In the next three sections, this paper gave a brief literature review from three perspectives: frameworks of thinking skills, assessment of pupils’ thinking skills, teaching thinking in Mainland China and its implication for educational equilibrium. Based on the literature review, research questions for this study were proposed.
1.1. Frameworks of Thinking Skills
There is no doubt that teaching of thinking aims to improve students’ higher order thinking skills rather than simple and rote memorization. However, as to what higher order thinking skills are, different theorists have different opinions. One of the most widely accepted definitions of higher order thinking skills (or higher order cognitive skills) is based on Bloom’s Taxonomy ( Anderson, Krathwohl, & Bloom, 2001 ; Bloom, Englehard, Furst, Hill, & Krathwohl, 1956 ). Bloom identified six fundamental hierarchical cognitive objectives, in which the top three levels (Analyzing, Evaluating, and Creating) are generally regarded as higher order thinking skills ( Anderson et al., 2001 ). Lewis and Smith (1993) defined higher order thinking as the thinking “which occurs when a person takes new information and information stored in memory and interrelates and/ or rearranges and extends this information to achieve a purpose or find possible answers in perplexing situations.” They listed some higher thinking skills, which include deciding what to believe, deciding what to do, creating a new idea, etc.
Another attempt was to define core thinking skills. Marzano (2001) identified a detailed list of core thinking skills which are defining problems, setting goals, observing, formulating questions, encoding, recalling, comparing, classifying, ordering, representing, identifying attributes and components, identifying relations and patterns, identifying main ideas, identifying errors, inferring, predicting, elaborating, summarizing, restructuring, establishing criteria, and verifying.
By conducting a meta-analysis of 55 frameworks, Moseley, Elliott, Gregson and Higgins (2005) devised a two-tier framework for learning and teaching thinking. This two-tier model distinguishes strategic/reflective thinking (i.e. engagement with and management of thinking) from cognitive skills (i.e. information gathering, building understanding and productive thinking). Moseley et al. (2005) used the terms “strategic and reflective thinking” here to reflect awareness and control not only of cognitive processes, but also of related motivation and affect.
Though differences exist in scope and in emphasis among theoretical frameworks for understanding thinking during the last half-century, some important common fundamental thinking capacities have been identified. These capacities are core thinking skills, critical thinking, creative thinking, problem solving, decision making and meta-cognitive processes across them ( Burke & Williams, 2008 ).
As explained above, the concepts of thinking skills, core thinking skills, higher order thinking skills, creative thinking, critical thinking, and meta-cognition are highly overlapping. In this study, we only distinguished meta-cognition from thinking skills and consider critical thinking, creative thinking and decision making as some sort of thinking skills. For each thinking skill, there is a corresponding meta-cognitive reflection.
1.2. Assessment of Children’s Thinking Skills
Extensive research has been carried out on assessing thinking skills. Most of these studies focused on assessing critical thinking ( Bissell & Lemons, 2006 ; Ennis, 1993 ; Gelerstein et al., 2016 ; Giancarlo, Blohm, & Urdan, 2004 ; Gorton & Hayes, 2014 ; Rickles, Schneider, Slusser, Williams, & Zipp, 2013 ; Saadati, Tarmizi, & Bayat, 2010 ; Stein, Haynes, Redding, Ennis, & Cecil, 2007 ), creative thinking ( Doppelt, 2009 ; Kim, 2006 ; Torrance, 1974 ) and reflective thinking ( van Velzen, 2004 ; YuekMing & Manaf, 2014 ). However, as Burke and Williams (2008, 2012) noted, none of these assessments integrated all thinking skills within one test. Moreover, few tests were intended for use with pupils.
Based on the thinking frameworks of Swartz and Parks (1994) and McGuinness et al. (2007) , Burke and Williams (2008, 2012) designed the “Assessment of Pupils’ Thinking Skills (APTS)” for pupils among 9 to 12-year-olds. APTS measures six thinking skills and corresponding meta-cognitive reflections comprehensively. In order to weaken the respective limitations of the multiple- choice tests and the open-ended tests, the APTS used a combination of these two formats.
The APTS measure is suitable for whole class testing ( Burke & Williams, 2012 ). Although articles that introduced the APTS measure had been cited more than 20 times, few studies showed that the measure had been used to investigate a relatively large sample of pupils. It remained a problem to get an overall developmental data of pupils’ thinking skills based on the APTS, and it was still unavailable to compare effects of thinking skills inventions in general.
1.3. Teaching Thinking in Mainland China and Its Implication for Educational Equilibrium
For a long time, education in China had been widely criticized for two defections. On one hand, teaching and learning was excessively focused on rote memorization rather than students’ thinking skills development. To reverse this situation, Chinese Ministry of Education launched the pilot of New Chinese Elementary Educational Curriculum Reform in 2001, in which simple knowledge instruction was replaced with a new Three-dimension Educational Objectives (i.e., knowledge and skills; procedures and methods; affection, attitude and values) ( Zhong, 2011 ). In 2010, issued by the State Council of China, Outline of China’s National Plan for Medium and Long-term Education Reform and Development (2010-2020) called for a new education reform that could prepare students for the 21st century. Under the guidance of education reform policies and supported with teachers’ continuing education, more and more schools and teachers have tried to shift their teaching from teacher and knowledge-centered to student- and thinking-centered approaches.
On the other hand, a serious imbalance in education development had received great concerns from publics. Making education more equitable has been regarded as a basic national policy in China Mainland. As stated in the Outline of China’s National Plan for Medium and Long-term Education Reform and Development (2010-2020), “The key of education equity is the equity of educational opportunities, in which the balanced development of compulsory education is a top priority.”
Studies on educational equilibrium development have boosted in recent 10 years. Most research on educational equilibrium focused on gender, urban-rural and interscholastic differences in terms of educational opportunities, public educational resources allocation, education quality and educational achievement ( Zhai & Sun, 2012 ). Literatures show that there has been a continuous development in equilibrium progress of basic education in China in terms of education opportunities, the distribution of educational resources, the educational quality and the educational attainment ( Zhai & Sun, 2012 ).
In fact, apart from differences among schools, differences also existed within schools. Controlling the balance within schools is more practical and feasible for principals. The development of pupils’ thinking skills is an important part of educational achievement. However, few studies concerned the differences among or within schools in terms of the development of pupils’ thinking skills.
1.4. The Present Study
This study was part of a larger study funded by MOE (Ministry of Education in China) as a project of Humanities and Social Sciences. The funded larger study was intended to promote educational equilibrium among primary schools through constructing a community of practice on teaching of thinking. In 2014, the community of practice, Alliance of Thinking Schools (ATS), was sponsored by the project team. Though most teachers and students were excited about their growth resulted from the teaching of thinking, obstacles always existed when they tried to ascertain the starting points or evaluate the effects of establishing their thinking skills interventions.
In order to get informed of the overall situation of the pupils’ thinking skills and to provide a relatively objective baseline for comparisons, this study investigated 2096 pupils in 4th, 5th and 6th grade from six primary schools in ATS. This study also aimed to explore the differences in pupils’ thinking skills development among schools.
The key research questions of this study are:
・ What was the overall situation of the pupils’ thinking skills in these six primary schools? How did pupils’ thinking skills developed over grades (i.e., 4th, 5th and 6th)?
・ Were there any differences in the pupils’ thinking skills among schools?
2.1. Participants and Context
The participants of this study were 2096 pupils (850 4th grade students, 676 5th grade students and 570 6th grade students) from six mainstream state-run schools in Beijing, Guangzhou and Xi’an, which located in North, South, and Northwest China respectively (Table 1). All of these six schools were members of the ATS. Since we were going to establish a thinking skills curriculum for students from 4th to 6th grades in these six schools, we chose students who were going to take thinking lessons as participants in this study. Before the tests, five of these six schools had not given any thinking skills interventions to their students. One school (#2) in Beijing had taught thinking skills fragmentarily in a school-based curriculum, the main intervention materials were Mind Mapping invented by Buzan and Buzan (1996) and five thinking tools (i.e., PMI, CAF, C&S,FIP, RULES) from CoRT1 designed by De Bono (1983) .
According to education policies in China, children under six years old are not allowed to go to primary school. For the school year started on September 1 every year, the ages of pupils in 1st grade range from 6 years old to 6 years and 11 months old, and the average is 6.5 years old when they enter schools in September.
Table 1. Participants profile.
As our investigation was carried out in March, which is the beginning of the second semester in the school year, the average ages of the pupils in 4th, 5th and 6th grades were 10, 11 and 12 respectively.
The instrument used in this study was the Chinese version of the APTS measure developed by Burke and Williams (2012) and translated by the research group. APTS can be used to investigate pupils’ thinking skills development with large-scale participants and/or assess effects of thinking skills interventions by monitoring changes in thinking skills over time among 9 to 12-year-olds ( Burke & Williams, 2008, 2012 ).
Six specific thinking skills were incorporated into the APTS (Table 2) ( Burke & Williams, 2008, 2012 ). In the APTS, the respondents are required to define the thinking skills and identify examples of the skills being used. Furthermore, the respondents are required to answer questions assessing how they apply the thinking skills and corresponding meta-cognitive reflection questions to identify the thinking steps they used to apply the skills in the previous questions. So, the APTS is comprised of three parts: Thinking Skill Definition (i.e., questions #1 , or D_TS for short), Thinking Skill Application (i.e., questions #3, #5, #7, #9, #11, #13, or A_TS for short) and Meta-Cognitive Reflection (i.e., questions #4, #6, #8, #10, #12, #14, or M_TS for short). The total score of Thinking Skill Definition, Thinking Skills Application and Meta-Cognitive Reflection for
Table 2. The structure of APTS.
individuals can be calculated by summing up questions #1 , questions #3,#5,#7,#9,#11,#13, and questions #4, #6, #8, #10, #12, #14 on each test respectively.
2.3. Testing Procedures
The investigation was carried out in March 2014, the beginning of the spring semester (or the second semester) in the 2013-2014 school year. As suggested by Burke & Williams (2012) , the tests were conducted in the pupils’ classrooms within 60 minutes. In the first five minutes, the printed questionnaires were handed out to the respondents, and respondents were asked to fill out some basic information, such as grades and classes. After that, the questionnaire was read aloud to the respondents by the testers in the classroom. Pupils who had difficulties in comprehending the items could ask for help.
2.4. Scoring and Data Analysis
Six junior or senior undergraduate students majored in Educational Technology at Beijing Normal University were trained by the researchers to rate pupils’ response papers according to the scoring matrix designed in the APTS ( Burke & Williams, 2008, 2012 ). Each response paper was rated by two of these raters (Rater A and Rater B) independently. A third rater (Rater C) checked the consistency of the scores given by Rater A and Rater B for each item. If the differences were within 1.0, the averages will be used as the final scores. Otherwise, Rater C would re-read the papers and gave the scores synthesizing the scores given by Rater A and Rater B. After finishing rating all the response papers, data were calculated with Excel and analyzed with SPSS18.0.
3.1. Reliabilities of the Scoring
To verify the reliabilities of the scoring by Rater A and Rater B, inter-judge reliabilities were calculated for each item using the Pearson product-moment correlation coefficient. It was found that the reliabilities ranged from 0.74 to 1.00 (Table 3). For item #1 and item #1, the inter-judge reliabilities were very high for these two items adopted multiple choice response formats. The reliabilities of others were relatively lower due to they were in the format of open-ended questions. So, a third rater was adopted to improve the accuracy and objectivity of scoring.
3.2. Overall Development of the Pupils’ Thinking Skills over Grades
3.2.1. Thinking Skills Definition, Application and Metacognitive Reflection
To present an overall situation of the pupils’ thinking skills from 4th to 6th grade, the average of D_TS, A_TS,M_TS and the total were calculated (Figure 1).
A one-way between-grades ANOVA was conducted to explore whether there were differences in scores of D_TS, A_TS, M_TS and total among grades. Results showed that statistically significant differences existed in all four conditions (Table 4).
Post-hoc comparisons using the Tamhane T2 test highlighted that the means of the total and A_TS of 5th grade were significantly higher than those of 4th grade were. Similarly, the means of the total and A_TS of 6th grade were significantly higher than those of 5th grade.
Table 3. The inter-judge reliability for each item of APTS.
**Significant at the 0.01 level.
Figure 1. Scores of D_TS, A_TS, M_TS.
Table 4. Mean performance of all three grades on D_TS, A_TS, M_TS and total.
For D_TS and M_TS, post-hoc comparisons indicated that statistically significant differences existed between 4th grade and 6th grade, and between 5th grade and 6th grade as well. However, no statistically significant differences existed between 4th grade and 5th grade.
3.2.2. Individual Thinking Skills
The APTS was broken down to analyze the differences in individual thinking skills among grades. A one-way between-grade ANOVA conducted to identify these differences showed a statistically significant difference in the mean of each skill among grades (see Table 5). Post-hoc comparisons showed that the skill CC and CUI significantly improved over grades. However, for the skill GRP, FRC, DM and PS, there was significant growth from 5th grade to 6th grade, while no significant gains were found from 4th grade to 5th grade (Table 5).
3.2.3. Meta-Cognitive Reflections on Individual Thinking Skills
The differences in meta-cognitive reflection on individual thinking skills among grades were also analyzed through a one-way between-grades ANOVA. Results showed statistically significant differences in the mean of all six metacognition skills among grades (Table 6). Post-hoc comparisons showed that the skill M_CC significantly improved over grades. However, for the skill M_GRP, M_CUI, M_FRC, M_ DM, and M_PS, though there were significant growthfrom 5th grade to 6th grade, no significant gains were found from 4th to 5th grade, and some meta-cognitive reflections (i.e., M_GPR, M_DM, and M_PS) even dropped slightly in this period (Table 6).
3.3. Differentiations in the Pupils’ Thinking Skills among Schools
3.3.1. Differences in Pupils’ Thinking Skills among Schools
To discover differentiations in the pupils’ thinking skills development among schools, the total scores, D_TS, A_TS and M_TS of each grade were analyzed
Table 5. Mean performance on individual thinking skills in three grades.
Table 6. Mean performance on meta-cognitive refection on individual thinking skills in three grades.
through a one-way between-schools ANOVA. For 4th grade and 6th grade, results showed statistically significant differences in the mean of the total, D_TS, A_TS and M_TS among all involved schools (Table 7). For 5th grade, results showed that there were significant differences in the mean of the total, D_TS, and M_TS among these schools, while there were no statistically significant differences in the mean of A_TS among these schools.
3.3.2. School-Characteristics of Pupils’ Thinking Skills Development over Grades
In order to explore how pupils’ thinking skills developed over grades within schools dynamically, one-way between-grades ANOVAs for five schools except school #1 were conducted respectively. School #1 was excluded for only students at 4th grade and 5th grade attended the investigation. Results showed that for all five schools, significant differences existed among all three grades (Table 8).
For school #2, #3 and #6, post-hoc comparisons highlighted that the mean scores of 6th grade were significantly higher than those of 4th grade and those of 5th grade, while there were no statistically significant differences between 4th grade and 5th grade.
For school #4, post-hoc comparisons highlighted that the mean score of 5th grade was significantly higher than that of 4th grade, while no statistically significant differences existed between 4th grade and 6th grade or between 5th grade and 6th grade.
For school #5, post-hoc comparisons highlighted that the mean score of 5th grade was significantly higher than that of 4th grade, and the mean score of 6th grade was significantly higher than that of 5th grade.
To make schools’ impacts on students’ thinking skills development over grades more explicit, line graphs were drawn to show dynamic trends of thinking skills development from 4th to 6th grade for school #2, #3, #4, #5 and #6 (Figure 2). It was obvious that these schools differed a lot from each other. For school #2, #3 and #5, students’ thinking skills developed over grades. However, for school #6, a slight decline was found from 4th grade to 5th grade. More surprisingly, for school #4, there was a significant decline from 5th grade to 6th grade.
Firstly, we will discuss the characteristics of the pupils’ thinking skills development from 4th to 6th grade, and then we will try to discuss the implication for teaching thinking in primary schools and educational equilibrium from the perspective of thinking skills development.
4.1. Overall Thinking Skills Development and Implication for Teaching of Thinking
4.1.1. The Pupils’ Overall Performance in Thinking Skills
Considering the full score is 72, the pupils’ performances in the test were relatively low to some extent. Take 6th grade for example, the average score (M = 28.96, S.D. = 5.15) did not reach half of the full score (i.e., 36.00), let alone the
Table 7. Mean performance of all six schools on the APTS.
Table 8. Mean performance of total scores of all three grades at each school.
Figure 2. Thinking skills development over grades in each school.
cut-off score (i.e., 43.20). When going deep into the three parts of the thinking skills, performance in meta-cognitive reflection (M = 8.36, S.D. = 2.10) was much poorer than that of thinking skills application (M = 14.23, S.D. = 2.51). Though pupils’ lack of experiences in answering such open-ended questions might affect their performance in the test, their incompetency in thinking skills and reflections should be blamed for this first. When pupils were asked to describe their thinking processes reflectively, most of them were at a loss. Education in China was widely scolded for its rote indoctrination and insufficient emphasis on students’ thinking processes. These results sent a signal that we should provide students with more space and time to think reflectively and to describe their thinking process more explicitly in daily teaching and learning.
In terms of individual thinking skills, students performed best in “Grouping” (M = 2.64, S.D. = 0.68 for 4th grade; M = 2.67, S.D. = 0.67 for 5th grade; M = 2.81, S.D. = 0.65 for 6th grade) and worst in “Coming up with ideas” (M = 1.73, S.D. = 0.62 for 4th grade; M = 1.83, S.D. = 0.62 for 5th grade; M = 1.95, S.D. = 0.68 for 6th grade). These findings were consistent with the characteristics of education in China, which focused much more on absorbing knowledge than discovering something new. As a result, knowledge on “What, Where, Who and When” were well taught in classes and knowledge about “How to generate new ideas” were usually ignored. For “Coming up with ideas” is an essential element of creative thinking, this finding urged that the focus of teaching should be shifted from rote reception to meaningful construction.
4.1.2. Development of Thinking Skills over Grades
4.1.3. Implications for Teaching Thinking in Primary Schools
The results enhanced the necessity and feasibility of teaching of thinking in 4th, 5th and 6th grade. For one thing, the fact that the pupils from School #2 outperformed others demonstrated that teaching of thinking, even fragmentally, was helpful to some extent. For another, the overall relatively lower performance in thinking skills development urged the importance to take effective measures.
4.2. Thinking Skills Differentiations among Schools and Implication for Educational Equilibrium
4.2.1. Thinking Skills Differentiations among Schools
Though similar distributions and trends of thinking skills development were found among these six schools, significant differentiations also emerged. This might result from different school cultures, teachers’ conception of teaching or students’ family background. Take school #2 for example, students in 6th grade performed much better than their counterparts in other five schools despite their urban-rural background. In fact, some thinking skills interventions (Mind Mapping and several tools from CoRT1) had been taught fragmentarily in the school since March 2013, which was one year ahead of our investigation. This was one possible reason why students at some grades there performed much better than students at the same grades in other schools did.
The performance of students in 5th grade from school #4 surprised us most not only for its leading place among these schools, but also for it was higher than that of 6th grade from the same school. However, the principal of this school was not surprised at all. She explained that teachers of this grade worked very hard and adopted latest educational ideas actively.
Though the differentiations among and/or within schools were comprehensive effects of many factors, it was schools’ administration and teachers’ teaching, other than economic situation or schools’ geographic positions, that played a more decisive role in pupils’ thinking skills development.
4.2.2. Implications for Educational Equilibrium
The results of this research revealed various differences of thinking skills development existed among schools. This reminds educators and administrators to pay more attention to educational equilibrium from the perspective of educational outcomes, especially students’ thinking development, rather than only in terms of educational opportunities and resources allocation.
4.3. Limitations of the Study and Future Research
This study indicated that the gain in meta-cognition and some thinking skills from 4th grade to 5th grade was relatively less than that from 5th grade to 6th grade. In order to acquire more details of thinking skills development during this period, it would be worthwhile to reexamine these differences in a qualitative way with the aids of interviews or case studies etc. It would also be interesting and valuable to verify which kind of interventions could be applied to improve these thinking skills, especially meta-cognitive reflections, and to what extent pupils’ thinking skills could be improved.
Moreover, as pointed by Burke and Williams (2008) , APTS captured thinking skills of students while ignoring other important aspects of thinking, such as thinking dispositions. Future studies could devise or adopt other measures to capture a broader scope of thinking.
This study investigated the pupils’ thinking skills in 4th, 5th and to 6th grade from six primary schools in Mainland China. Findings indicated that pupils’ thinking skills grew dramatically over grades; however, these skills grew much more slowly from 4th to 5th grade than that from 5th to 6th grade. Findings also suggested that differentiations existed in pupils’ thinking skills development among schools. To explore the characteristics of the pupils’ thinking skills development, this study made an important contribution to broaden the scope of application of APTS in Chinese condition, and provided a relative objective baseline for examining effects of various thinking skills interventions and for comparing students’ thinking skills development in different areas as well.
The study was supported by a MOE (Ministry of Education in China) Project of Humanities and Social Sciences in 2014 [grant number 14YJC880117]. The authors would like to acknowledge Dr. Lynsey A. Burke for allowing the APTS tests to be used, teachers and students from these six elementary schools for their assistance in the investigation, Professor Simone C. O. Conceição and reviewers for their valuable comments, and Dr. Jacqueline Madhok for proofreading the paper.