Received 8 January 2016; accepted 26 April 2016; published 29 April 2016
Today public education in many European Union member states is struggling with poor learning motivation experienced among students. The consequent poor scholastic achievement (OECD, 2013) is an acute problem across Europe, particularly in institutions where the majority of students are socially disadvantaged.
A successful way to tackle this problem is application of the methodology of the so-called Creative Partnerships (Creative Partnerships, 2007) programme (see Sharp et al., 2006 , Office for Standardsin Education, 2006 , Eames et al., 2006 ).
Developed in the United Kingdom, it has been embraced by several countries, currently in Lithuania (Creative Partnerships Lithuania, s.a.) , Norway (Creative Partnerships Opland, s.a.) , Germany (Gabriele Fink Stiftung- Kulturschule, s.a.) and the Czech Republic (Creative Partnerships Prague, s.a.) . In Hungary, the programme has been piloted since the 2013/14 school year with the collaboration of the Faculty of Music and Visual Arts of the University of Pécs and T-Tudok Centre for Knowledge Management and Educational Research Inc.1 in seven member schools belonging to Budai Városkapu (“Buda City Gate”) Primary School, Vocational School, Special Vocational School and Basic Level Art School of Pécs. The programme has continued successfully since. So far a total of 420 students, 42 teachers and 14 creative practitioners have been involved in two stages. In 2015 the Hungarian Educational Authority admitted the Creative Partnerships programme to feature among the so-called regional educational model programmes.
A Creative Partnerships programme is focused on the long-term relationship between creative practitioners and schools. Creative professionals from various branches (of arts) help students master different types of knowledge and skills. Creative practitioners bring new expectations, which clearly challenge students. But it is not only students that are jerked out of the rut of daily routine by the new learning process―schools as a whole are also affected. Creative Partnerships has developed a pedagogical approach known as the “high functioning classroom” which encourages teachers to change their classroom practice so that lessons are replete with challenges that relate learning to real-life situations; where students are engaged physically and socially, as well as emotionally and intellectually; and where students’ own experiences, observations and questions take centre stage. The transformation of teaching and learning is based on creative processes which are channelled into classrooms and school life. Techniques applied in the methodologies of teaching various subjects are derived from artistic practices. Whilst working in the classroom with the students the artist remains an artist, and the teacher remains a teacher. The programme does not offer arts education. What the Creative Partnerships programme does is to prepare artists to work with teachers and students to bring about sustainable changes in teachers’ teaching practice.
The classroom work of creative practitioners has recently been studied in depth by researchers of Cambridge University (McLellan et al., 2012) and Nottingham University (Thomson et al., 2012) . Their reports point out unequivocally that creative practitioners apply effective educational techniques that teachers can easily adopt and immediately incorporate into their day-to-day practice. This is the pedagogical approach that lies behind the measurable change in the motivation, achievement and conduct of students proven by the studies.
1.1. The Educational Concept of the Creative Partnerships Programmes
The notion of “high performers,” i.e. “high functioning” students highlights why the creative approach has a positive effect on students’ engagement and academic attainment. The evaluation (McLellan et al., 2012) of the England-based Creative Partnerships programme demonstrates that a student educated in a context in which they are an essential learning resource, and where mobility, emotion, team working and risk are central to the learning experience, is a student who is “high functioning”. In this form of education, the whole child is engaged in the learning experience, not only aspects of their mental processes, but their bodies, their emotions and their social skills. It is this sense of being “high functioning” which leads to feelings of well-being within the child, and this in turn builds the resilience and confidence which underpins successful learning. These practices are effective because they directly impact on the students’ sense of competency, autonomy and relatedness. They provide the sense of agency and motivation from which sustainable learning power is generated. This is why researchers found students at Creative Partnership schools to be more engaged, better behaved and achieving more. As Maurice Galton explains:
“In this model of educational progression children move from a point where they acquire knowledge that is already known by others, to a point where they can order that knowledge within particular frameworks, to a further point where they can, without too much assistance, interrogate their own thought processes in creating their personal frameworks or restructuring existing” (Galton, 2007: pp. 5-6) .
Figure 1 shows the characteristic features of the high-functioning classroom.
1.2. Operation of the Hungarian Pilot Programme
In Hungary the artists and art students participating in the programme are selected by the staff of the Theory and Art History Department of Pécs University’s Faculty of Music and Visual Arts2. They complete a 40-hour training course which incorprates training of the teachers from the participating schools. The training sessions are led by experienced trainers of the British host organisation Creativity Culture and Education (CCE)3. Exploration of the institutional problems of the schools and preparation of problem maps are led by T-Tudok experts. One form or student group of one or two years per school are involved in the programme, which means that the so-called local mini-projects are conducted by two trained teachers and two creative practitioners, each pair working in close cooperation. The mini-projects focus on tackling local difficulties highlighted by schools and teachers (e.g. reducing the risk of attrition, development of cooperative skills, managing disruptive behaviour, strengthening of a positive image of the future, improving the “visibility” of the school in the local community) or school discipline specific knowledge transfer and trans-disciplinary competence development. After an on-site observation session and two planning sessions the local projects were implemented in 10 to 12 three-hour (3 × 45 minutes) modules, and were concluded by detailed evaluation and presentation. The modules can be incorporated in the structure of curricular classes as well as school-based extra-curricular afternoon sessions.
1.3. Stage 1 of the Pilot
The first stage of the pilot took place in the spring semester of the 2013/2014 school year and involved all seven schools belonging to the Budai Városkapu School Centre. It was of significant help that the institution chosen
for the pilot was a complex school centre whose members included a school with only lower grade (years 1 to 4) classes as well as a special vocational school and an eight-grade basic level art school. This multifaceted complexity lent itself to trying the programme in various contexts and differing learning environments, in classes of teachers who used different methods of learning/teaching organisation. Rather than focusing on transferring contents in various school subjects, Stage 1 was targeted at solving/resolving the main problem perceived by the teaching corps of each school with the particular student community that teachers felt they had the biggest difficulty engaging. The sessions generally took place in the afternoon, in a curricular or extra-curricular context.
Effectiveness was measured by means of student questionnaires and qualitative methods of assessment. The findings show that the local projects had a positive impact on all stakeholders: the students and their parents, the teachers, the schools as well as the creative practitioners alike. The general opinion of school staff is that the programme triggered a positive development in the institution: a significantly more student-centred approach has developed and opportunities have opened that they had never considered before. The performance of the participating students significantly improved, and the achievement of socio-culturally disadvantaged students surpassed the nationwide average. In the wake of their children’s accounts parents have become increasingly curious about the programmes; wishing to be included in school life and, through the programmes, becoming increasingly motivated about their children’s progress. As for the creative practitioners, i.e. the participating artists and art students, they have become leaders and implementers of community art projects.4
1.4. Stage 2 of the Pilot
The second stage of the programme was launched in the spring semester of the 2014/2015 school year in the context of the Educational Authority’s priority project SROP-3.1.1-11/1-2012-0001 21st Century School Education: Development and Coordination (Second Phase), Creation of Innovative Networks and development of Educational Programmes. The Creative Partnerships programme appears in the project as a possible model of a network of regional partnerships whose goal it is to improve sub-par performance of schools in mathematics―a problem that does not involve nationwide networking but concerns the teachers of more than one institution. The programme was implemented with the approval of the Pécs School District and the full support of the heads of participating member schools. After consultations with the heads of schools three schools were selected with a Grade 5 class in each to serve as pilot sites. A Grade 5 class of another member school and one from a pilot school were chosen as control classes; the latter had never been involved in, and the teachers were not familiar with, the Creative Partnerships programme. The sociocultural and demographic indicators of the students were the same as those of the pilot classes.
Training for teachers and creative practitioners was delivered at the Faculty of Music and Visual Arts of the University of Pécs from 26 - 28 January 2015. This training was able to be shorter than CCE’s normal practice, because the practitioners and teachers were already familiar with the programme. The practitioners had all participated in the first stage of the programme a year earlier. The training was particularly focussed on the delivery of maths
Activity began in schools in February and was focussed on a single class of Grade 5 students in each school. There were weekly 90-minute sessions in each school until the end of May. The creative practitioners again worked in pairs, so three pairs of creative practitioners, i.e. six artists (painters and sculptors) collaborated each with a pair of teachers (one maths teacher and one form teacher). There were altogether five fifth-grade classes involved. There were a total of 46 students in the pilot classes and 47 in the control classes. Only 88 students filled the test of the qualitative analysis at both the input and output stage, therefore only these tests have been analysed.
The internal (qualitative) evaluation of the programme took place in April. In each school the assessors observed the maths workshops designed by the creative practitioners and the teachers, and they were able to talk separately to the schools directors, maths teachers, creative practitioners and a selection of the children.
1.5. Main Findings of the Qualitative Analysis
In all the classes observed, and from the discussions with the creative practitioners and the teachers involved in the pilot, it was clear that new ways of working, talking, thinking and assessing had been introduced. This was evident in a number of ways:
・ Teachers and creative practitioners had experimented with using different lesson lengths. In the most successful cases, lessons had lasted 90 minutes, and were able to retain the attention and focus of the children over this period. It was noticeable, however, that children had taken time to adjust to the longer lesson times, and to begin with there had been behavioural problems.
・ In planning the lessons, teachers and artists had been careful to give time in every session to “community building.” In one of the schools, for instance, the session began with an exercise in which the students stood in a circle and had to pick a name of a pupil out of a hat. Then each pupil had to say which flower they thought that pupil whose name they had picked and give a reason why. All the students were generous in their selections and explanations, generating a positive atmosphere in the class. In addition, it allows each pupil to have individual attention from the whole group. Building a class community, which builds on the individuality and diversity of each child, is a very important element in building a collaborative learning environment in which every child has the opportunity to thrive.
・ All the lessons observed, and the descriptions provided of other lessons delivered, used space both within and outside the classroom flexibly and imaginatively. All the teachers reported that designing more physically active maths lessons had improved concentration and learning. In one school, children were given big sheets of paper and asked to draw outlines of parts of their bodies which they then measured and used for other calculations. One girl, who generally remains detached from maths learning, became very absorbed in this task, experimenting with different ways of completing this task in a remarkably inventive way. This allowed her to explore the idea with her curiosity and imagination combined.
・ All the lessons observed had allowed time for effective reflection. In some cases this allowed children to repeat for the benefit of the whole class elements of the work they had done in smaller groups, hence reinforcing the learning and ensuring that the main concepts had been understood. Generally these discussions ensured that the teachers had a good grasp of how much learning had taken place and highlighted any work which would need to be reinforced later. These sessions also contributed to building the class community. It was interesting to observe in one of the schools how long a big class (22 students) were able to sustain the reflective discussion without becoming restless.
・ The activities all showed that care had been taken to design in challenge, collaboration, authenticity, inclusiveness, mobility and flexibility. For instance, in one school, a 90-minute lesson was observed. The lesson began in the classroom with a warm up exercise. The teacher then introduced the main exercise of the day. The pupils were divided into four groups. They were told that round the school, both inside and outside, four “stations” had been established. At each station they would find an envelope containing a set of instructions. They had to complete the exercise described in the instructions, and continue until they had completed all four exercises. Although at each station there was an adult. These adults played no role in directing the children but they were prepared to answer questions if there was anything that the pupils had not understood.
・ Working in small groups of four or five allowed every student to play a part, an opportunity which all the children in the groups seized with enthusiasm, ensuring that the lesson was inclusive. Moving around the school building ensured that the lesson was highly mobile. Although they were doing a lot of maths exercises, like working out mean averages, none of this work was done sitting at a desk. The overall design allowed children to select how they went around the task, in what order they should be completed, and to work at their own pace ensuring great flexibility.
・ The behaviour of the pupils, in particular their focus, discipline and resilience, was impressive. One group collected their instructions and after a brief discussion set about their task. However, after a while it became clear to them that the approach they had agreed on was not working. They were able to discuss this among themselves and develop a new approach to the task. The only help they had from an adult was to ask whether it was allowed to begin again-permission they were granted. This was particularly impressive because the children in this Grade 5 class were able to demonstrate that they could, without assistance, progress from a point where they acquire knowledge, to a point where they can order that knowledge to a further point where they can interrogate their own thought processes to create their own personal frameworks or restructure existing ones. This is a high level of cognitive ability which the design of the exercise allowed them to develop and display. The structure of the control lesson that was observed, on the other hand, would not have allowed this aspect of their thinking to be developed.
・ All the teachers interviewed for the evaluation reported that they had welcomed the many new ideas that the creative practitioners had brought to the planning of lessons and that they already had or planned to use these exercises and approaches themselves.
・ The principal of one of the member schools acknowledged that she had taken one of the ideas she had seen in a lesson designed by the creative practitioners and was now using this herself in her own teaching. In this exercise, the pupils stand in a circle. Each of them has chosen a number. In the middle of the circle there are numbers written on cards and laid out on the floor. The pupils have to group themselves so that the total of the numbers of each child in the group equals one of the numbers on the floor.
・ The maths teacher in one of the pilot classes said she very much liked the exercises that the creative practitioners had introduced and planned to use the exercises herself in the future. She admitted that her approach to maths was very “logical” but that she had been amazed at how the unusual approaches introduced by the creative practitioners had helped the students learn some concepts much more quickly.
・ In another pilot school the teacher explained that one lesson she had planned with the creative practitioners had to be postponed because it required going outside and that the weather on the day made this impossible. The following week, the creative practitioners were unable to come, so the teacher decided to try it on her own. The exercise required the children to work in pairs and to search the grounds of the school for 15 hidden envelopes. Each envelope had a mathematical problem to be completed. The teacher reported that the whole exercise went really well. Not all the children completed all 15 exercises, but they all remained focussed and engaged throughout the time. The teacher commented that all she had to do was sit on a bench and watch as the students did all the work. She said that this had given her great confidence and that she planned to develop further exercises and complete them on her own.
All the teachers, school directors and creative practitioners reported that there had been improvements in the students’ behaviour. The teachers were amazed that the students had been able to remain focussed and disciplined throughout the 90-minute lesson given the degree of mobility and independence that the students had been allowed.
1.6. Methods and Tools of the Quantitative Analysis of the Mathematics Pilot Programme
The goal of the quantitative analysis was to measure the impact of the second stage of the Creative Partnerships pilot programme on certain competencies, reading skills development and maths knowledge of the participating students. The analysis addressed the following areas:
・ Social competence
・ Motivation to learning
・ Reading level
・ Mastery of certain concepts and areas in mathematics
This analysis explores the results of 88 fifth-grade students from five classes of the seven member schools of Budai Városkapu Primary School, Vocational School, Special Vocational School and Basic Level Art School of Pécs.
The assessment was conducted in the spring semester of the 2014-2015 school year over a period of three months, from March through May. In the first, 2013, stage of the programme T-Tudok Inc. only sent participants a questionnaire after the completion of the project. In the second stage, students were surveyed at both the input and the output ends, which allowed the assessors to measure the change between the two points.
In order to analyse the impact of the Creative Partnerships pilot programme, a control population had to be designated consisting of students with similar background and competencies who are not involved in the programme in any way. It is important to have a control group with which to compare the pilot students’ progress because at this age students’ social skills, learning attitudes and scholastic achievement naturally change. If only the pilot group were analysed it would be impossible to determine whether any change in the areas examined can be attributed to participation in the pilot or if it is the result of some external or age related impact.
The 88 fifth-graders were divided into a pilot and a control group, with 44 students in each group. The above mentioned six areas were analysed not only in terms of pilot and control groups; both groups were broken down further by family income and type of educational institution. In this way it was possible to get a picture of which student group is affected most by the programme, and whether the impact on students is consistent across various school types.
The following tests were used to measure the impact of the Creative Partnership programme on students’ performance.
Reading skills are measured by the reading level test developed by the Hungarian Institute for Educational Research and Development (HIERD) and administered to Grade 4 students as part of the National Test of Educational Skills and Competency Standards5. This part of the analysis is therefore comparable to the result of the nationwide reading skills test.
The mathematics test was developed by mathematics experts invited by T-Tudok.6 The tests applied at the input and output stages were aligned with curricular requirements. The tasks set addressed four major areas: arithmetic, measurements, geometry and problem solving. The input test measured children’s numeric skills, knowledge of measurements, solution of geometric operations, and tested their ability to analyse problems. The output test explored children’s knowledge of geometric concepts, their numeracy in the set of integers, as well as their skills to interpret simple charts and graphs and use of coordinates.
In addition to reading and mathematical skills changes in the students’ self-evaluation and personality were also investigated. The questionnaire used had previously been applied by HIERD in the assessment of the day school programme (SROP 3.1.1. Second Phase, Subproject 3: Complex Art Education Component, 2012) . It explores several areas including self-image, social competence, learning motivation and empathy. T-Tudokhad earlier administered this test in other classes of the schools analysed below, therefore it is possible to compare the attitudinal changes described in this report to the findings of a previous study and to a nationwide research project.
Given the very small number of components in the database, our findings are not suitable for generalisation or for observing trends of improvement or deterioration. Nonetheless, the database provides valuable information about the programme’s functioning and mechanisms of impact.
2. Findings of the Student Assessment of the Mathematics Pilot
2.1. Self-Image, Social Competence, Empathy, Learning Motivation
This chapter addresses the changes in the pilot and the control group students’ answers to attitude and motivation related questions between the input and the output surveys.
Social attitudes are analysed in four areas: self-image, social competence, empathy and learning motivation. They have been assessed by means of 34 items, and each area was addressed by a variety of questions. Each question involved four possible answers: it’s not at all like me, it’s not like me, it’s like me, it’s very much like me. The first two answers were considered negative: if the student chose either of the two, then the statement does not characterise them. Likewise, the other two answers were considered positive: students choosing these options think that the trait or behavioural pattern formulated by the statement is typical of them. In the analysis the researchers counted the number of positive and negative choices in each case. The small number of respondents did not allow multi-variable analysis; nevertheless, these techniques are worth applying also in larger samples.
2.1.1. Main Findings of the Analysis
・ In the pilot group the students’ positive self-image has improved to a greater extent than in the control group.
・ Pilot students’ positive attitude to social competence has also improved to a greater extent compared to the control group.
・ The levels of empathy and learning motivation have not increased significantly in the pilot group or in the control group.
Ten statements were used to test students’ self-image. It is conspicuous that the number of students making only few (1 - 6) positive statements decreased in both the Creative Partnerships (CP) pilot group and the control group, while the number of those who felt that 9 or 10 positive statements described them increased. Figure 2 shows that while in the control group there was a 7% drop in the number of those who gave few positive answers and the number of those who felt almost all statements described them grew by 16%, the same rates in the pilot group were 33% and 41% respectively. This indicates a stronger positive process among the CP pilot students.
2.1.3. Social Competence
Figure 3 shows that a similar positive trend seems to emerge in respect of social competence, which is stronger in the pilot group than in the control group. The number of students who felt they were characterised only by 0 - 6 statements was down by 42%, and the number of those who gave a positive answer to almost all statements was up by 160%. In the control population the number of those in the lowest category did not decrease but 67% more students’ answers were almost all positive. It is to be noted that the control group was very positive upon input, so more students identified with 8 - 9 statements upon output.
Figure 2. The number of positive statements relating to self-imagein the input and out- put tests.
Figure 3. The number of positive statements relating to social competence in the input and output tests.
Four statements were applied to test empathy. As shown in Figure 4, the findings are not quite as straightforward as in the case of the other two indicators. In the pilot group three students less gave a positive response to 2 or 3 statements in the output test that upon input, and the number of students giving the most positive answers grew by one, while the number of students giving the fewest positive answers increased by two. In the pilot group the number of those who felt they were characterised by the most as well as by the fewest statements. The findings indicate a slight decrease in the empathy of the pilot students and a very minor increase of empathy in the control group.
2.1.5. Learning Motivation
Learning motivation showed an evident, albeit slight, decline in both groups, as presented in Figure 5. In the pilot group 12.5% fewer students said 7 or 10 statements described them, and the decline in the control group was 8%. In the pilot group the only increase was in the number of those who identified with the fewest number of statements. In the control group the number of students giving 0 - 4, as well as 5 - 6 positive replies increased.
2.2. The Relationship between Social Background and the Factors Examined
The input test took stock of the students’ social situation. This enabled the researchers to analyse indicators described above in subgroups based on the income of the families. This is important because the picture about the impact of the programme is more accurate if not only the overall effects on the group as a whole are considered but other questions can also be answered, for example whether the programme managed to enhance the learning motivation or the maths performance of students coming from the poorest families, or the empathy of students from the most affluent families.
To plot students’ social background the researchers asked whether the family had certain assets (five were listed) and whether the student personally had certain possessions (four were listed). The longer list of 13 - 15 items generally used in other studies was not applied because according to our preliminary information the majority of students in the schools came from families in difficult social conditions. As regards family assets, the list contained a computer, a car, a washing machine, a dish washer and a holiday home. The personal possessions list included an own room, an own desk, an own mobile phone and an own bicycle. As can be seen, neither list investigated the possession or absence of particularly expensive items. For example, in the case of a child not having an own desk indicates serious deprivation as a desk is an indispensable material condition for learning.
Figure 4. The number of positive statements relating to empathy in the input and output tests.
Figure 5. The number of positive statements relating to learning motivation in the input and output tests.
Possessions give an idea of the family’s social background. The more items a family and the child in the family have the more affluent they are. Contractions were made for the sake of analysis. In terms of family assets, efforts were made to set up groups representing a quarter of the families, and in terms of personal possessions, each group represents one-third of the children.
The following sections concentrate on the number of positive answers students gave in respect of their self-image or empathy through the evaluation of the statements, and the number of correctly or incorrectly solved or unsolved maths exercises and problems as well as reading comprehension tasks in the input and output tests. For ease of comparison most of the results are presented in a standardised format.
To illustrate this by the example of maths questions, standardisation means that we look at the number of correct answers each student gave to the maths exercises or problems. These values are then cumulated and standardised, i.e. transformed in a way that their mean is zero and their spread is one, so their value is most frequently between +1 and −1. This means that if a standardised value is a negative number, the group gave less-than-average correct answers, and vice versa, a positive value means students performed above the average. If a positive value appears in the social competence analysis, it means the group identified with more than the average number of statements. The input and the output results were standardised separately.
It is not the standardised results themselves that the analysis is focused on but their change between the input and the output tests. Positive changes are marked in green and negative changes in red in the output table. If a change was less than 50% the cell remained white.
Comparison with previous results
In order to see our findings in context and compare them to the findings of recent surveys in Hungary the results of several research projects are presented in the following passages.
The basis of comparison of social attitudes and competence was the test administered by T-Tudok to seventh-graders of the schools participating in the 2014 Creative Partnerships pilot, and HIERD’s impact analysis carried out in conjunction with the development of educational programmes in day schools.
Reading level was compared to the Educational Authority’s National Test of Educational Skills and Competency Standards conducted in 2012 among Grade 4 students.
2.2.2. Main Findings
Social attitudes and competence
・ Except for the lowest income group the CP pilot had a significant positive effect on students’ self-assess- ment and self-image.
・ The empathic skills of students decreased in both the pilot and the control group. In the pilot group the empathy of students with the most modest background also deteriorated, which may widen the gap between the two groups.
・ Participation in the pilot had a positive effect on the learning motivation of students from affluent families but had no impact on the motivation of the poorest students.
・ The better the students’ living conditions the more they tend to give positive answers.
・ Based on the two 2014 tests used for comparison, more affluent students tend to have a more positive self-image and better learning motivation; conversely, they have less empathy than their peers from poorer families.
・ The rate of students with optimal reading skills increased significantly in the pilot group, to an extent exceeding that of the control group.
・ Students in the pilot group produced more correct answers to the maths exercises and problems that the control group students. The increase was conspicuous mainly in the group of affluent students.
・ There was a slight increase in the number of incorrect answers in the pilot group and a parallel decrease in no answers. This can be explained by increasing mathematical confidence: in the output test students attempted to solve a problem even if they were not sure of the solution. The findings show that confidence increased not only among the children of the most affluent families but across the board.
2.2.3. Analysis of Social Attitudes and Competence
The findings show that the family’s assets have a strong influence on the child’s personal possessions. Table 1 shows that in 41% of the very poor families (with 0 - 2 items from the list) children too have only 0 - 2 items of possession. The other extreme is just as strong: in 92% of the families having all five items the child has four items of personal possession.
It is conspicuous that families strive to give their children more personal items than the family has as a whole. Sixty percent of the families with the least assets provided more than two personal possession items to their children. The rate is similar for families possessing three and four items. This, however, also means that a family may have some items but the student does not. Each of the first three categories of families contained at least two children who do not have a desk of their own. These students are severely limited in their home studies.
The average numbers of positive answers by family possession were also examined. The results presented in Table 2 and Table 3 show that the CP pilot programme had a significant positive impact on the participating
Table 1. Distribution of family and personal possessions (percentages in brackets).
Table 2. Distribution of standardised average positive answers by family possessions upon input.
*Very low case number 1 case per cell.
Table 3. Distribution of standardised average positive answers by family possessions upon output.
*Very low case number 1 case per cell.
students’ self-evaluation and self-image, except for the lowest income group where the impact was minor. The empathic skills of affluent students in the pilot group as well as the control group decreased―it seems that the programme was unable to offset the negative trend. Participation in the programme had a positive effect on the learning motivation of affluent students but did not affect the poorest students.
It is interesting to note the Total lines of the tables. Apparently the control students had a more positive attitude and self-image at the beginning of the pilot period and deteriorated by the time of the output test. The pilot group displays the opposite trend: the students’ self-image and social competence changed positively. Conversely, in terms of empathy and learning motivation the control group’s initial positive results improved further and the results of the pilot group, already low at the input test stage, deteriorated further.
Similar conclusions can be drawn from the analysis based on the students’ possessions shown in Table 4 and Table 5. It should be noted that unlike the control group, the level of empathy of pilot students with the most affluent as well as the poorest background diminished, which is likely to widen the gap between the two groups.
Both breakdowns highlight the fact that the better the circumstances of the students the more likely they are to give positive answers, i.e. the more they identify with the statements used in the tests.
Comparison with previous research findings
The above results were compared with an earlier survey conducted by T-Tudok Inc. in 2014 in Grade 7 of the Pécs schools involved in the current CP programme. The results of the survey are presented in Table 6 and Table 7.
Table 4. Distribution of standardised average positive answers by personal possessions upon input.
Table 5. Distribution of standardised average positive answers by personal possessions upon output.
Table 6. Distribution of standardised average positive answers by family possessions in the 2014 T-Tudok survey.
Table 7. Distribution of standardised average positive answers by personal possessions in the 2014 T-Tudok survey.
The processes here are clearer than in the 2015 impact analysis. The better the family’s financial situation the higher the distribution of standardised positive answers in the case of self-image, social competence and learning motivation. On the other hand, empathy deteriorates as financial conditions improve.
Our findings were compared with those of HIERD’s 2014 impact analysis carried out in conjunction with the development of educational programmes in day schools. Table 8 and Table 9 present HIERD’s findings by income categories.
These two tables reflect our findings. The more affluent family students come from the more positive their self-image and the stronger their motivation to learning. However, in HIERD’s analysis more affluent students
Table 8. Distribution of standardised average positive answers by family possessions in the 2014 HIERD survey.
Table 9. Distribution of standardised average positive answers by personal possessions in the 2014 HIERD survey.
not only have poorer empathy but also poorer social skills.
2.2.4. Reading Level
According to definition applied by the Educational Authority, “The critical preconditions for a well-functioning reading comprehension are optimal reading skills and knowledge of the meaning of the 5000 most frequent words as 96 percent of the words of standard written texts are variations of these words.” Table 10 presents the current level of mastery of this basic vocabulary among fourth-graders based on 2012 results7 (OKM, 2012: p.5).
The values in the table serve as reference for our analysis. The average level of reading skills in the Grade 5 group in our analysis is 84 percent. As Pécs is a county seat the reference value is 87 percent, a difference of three percentage points. This does not seem to be a major difference but as our target group is a year older it can still be significant.
Figure 6 presents the input and output reading levels of all of the Pécs students involved in the analysis. It is conspicuous that in our analysis the rate of fifth-graders with advanced and finishing level reading skills is considerably higher than the reference figures in the table. Conversely, the rate of students with optimal reading skills is lower (16% compared to 33%), which is considerably below the nationwide average given that the students in the Creative Partner ships programme were one grade older.
Figure 7 presents the reading levels of students in the pilot group and control group separately at the time of the input and output testing. An important outcome of the programme is that in the pilot group the rate of students with optimal reading level increased considerably, to an extent exceeding that of the control group: the rate of increase in the pilot group is 350% as opposed to 25% in the control group. Concurrently, the rate of advanced level readers dropped drastically in both groups.
Table 11 and Table 12 address the relationship between the level of reading skills and the average number of positive answers to attitudinal questions. At the input stage there is a clear relationship: the better the reading skills the higher number of positive answers. This correlation does not prevail at the output stage, where a significantly greater number of students were found to have improved reading skills. It is also conspicuous that while the number of optimal readers increased this did not go hand in hand with an improvement of their social competence, empathy and learning motivation: these indicators did not improve.
2.2.5. Mathematical Competence
In this chapter the results of the mathematics components of the input and output tests are analysed. Three variables have been set up based on how many students gave how many correct answers, incorrect answers or no answer at all. The figures were standardised as described in the Methodology chapter above. Positive figures mean that a group gave more than the average number of correct or incorrect answer or no answer.
The input test included 21 maths exercises and problems. Table 13 is presented to illustrate the methodology by means of the exercises/problems included in the input test. The columns show the numbers of correct answers. Thus a value 4 means that the student solved four exercises/problems correctly. The rows show how
Figure 6. Reading levels of the participating students in the input and output tests.
Figure 7. Reading levels by pilot and control group in the input and output tests.
Table 10. Reading level.
many children in the pilot and control groups gave how many correct answers. So in the pilot group 1 is attached to the value 4, which means that there was one student who gave four correct answers, and in the control group four children gave four correct answers.
Figure 8 reveals that on the average students in the control classes gave incorrect answers more frequently than the children in the pilot classes. Moreover, the difference increased by the time of the output testing, which meant that as a result of the programme students in the pilot classes solved maths exercises and problems correctly with greater frequency.
Figure 8. Average distribution of maths answers in the input and output tests.
Table 11. Distribution of standardised average positive answers to social skills by reading level upon input.
*Very low case number 2 cases per cell.
Table 12. Distribution of standardised average positive answers to social skills by reading level upon output.
*Very low case number 2 cases per cell.
Table 13. Distribution of correct answers in the input test.
In the pilot group the average number of correct answers increased, that of missing answers decreased while the number of incorrect answers somewhat increased. This can be explained by the pilot group students’ greater mathematical confidence: they dared to try and solve the problems and exercises even if they were not quite sure how to solve them.
Table 14 and Table 15 explore the correlation of maths answers with the students’ social background. The results of the output test indicate the frequency of correct answers grew significantly among the students of the most affluent families. At the same time, missing answers decreased even among students with poorer backgrounds, which means that they too showed greater confidence in trying to solve the problems.
As one of the pilot classes comprised children with special educational needs their performance could arguably influence the impact of the programme. However, disregarding SEN children’s impact does not change the results to any great extent. Therefore on the whole, SEN students do not have a significant influence on the results.
In addition to the above, mathematical skills were examined in the individual areas as well. Maths tasks addressed four major areas: arithmetic, measurements, geometry and problems.
As the two tests included different types of exercises and problems and not all the areas overlapped it is not possible to directly compare the input and output results summarised in Table 16 and Table 17.
Table 14. Distribution of standardised mean maths answers by family possession upon input.
*Low case number 1 case per cell.
Table 15. Distribution of standardised mean maths answers by family possession upon output.
*Low case number 1 case per cell.
Table 16. Standardised mean values of some areas in mathematics upon input.
Table 17. Standardised mean values of some areas in mathematics upon output.
Based on the input as well as the output tests students in the pilot group perform better than the control group in three areas, and their achievement was poorer in solving problems in the first test, and in working with graphs and charts in the second test.
The maths tests were evaluated by a mathematics teacher, who calculated each student’s result in percentages for the overall test and also for each mathematical area tested. The results contain a certain amount of subjectivity as the tests were evaluated by a teacher who only knew one of the classes involved. The overall performance and the achievements in the four respective areas are depicted in the following tables (in percentages).
Table 18 and Table 19 show that the performance of both pilot and control students was relatively poor in both tests, the overall achievement remained below 50%. However, in harmony with our earlier observations, students in the pilot group managed to significantly improve their overall achievement, to a much greater extent than the control group students.
On the whole, as a result of the programme the maths performance of the students in the pilot group improved considerably. Moreover, besides their knowledge, their confidence also increased, and at the output stage they tried to solve more problems and exercises than in the input test. Here again, family background plays a crucial role and the Creative Partnerships programme did not manage to stop the exclusion of the poorest groups.
3. Main Findings by School
In this chapter the findings related to the investigated areas of social attitudes and competence, reading level and mathematical competence are broken down and analysed by school.
Table 20 presents the distribution of students in the control and pilot groups across the member schools of the Budai Városkapu School Centre of Pécs:
The main findings of the analysis are as follows:
・ The pilot students in the different schools mostly seem to move in the same direction in terms of the various indicators; it is likely that the programme has a consistent impact that is independent from the institution.
・ The students’ self-image in the pilot schools significantly improved, to a greater extent than the control group. The biggest improvement was observed in the case of the pilot students in Meszes School.
・ The social competence of pilot students also improved. The most marked positive change was found with the Somogy School students, where the rate of respondents identifying with almost all of the statements grew from 0% at input to 38% at the output test.
・ The level of empathy of students decreased in all of the schools with the exception of the Bártfa Street institution, where the number of students who identified with all of the statements was up from 11% to 33%.
・ With the exception of Meszes School the rate of students giving a positive answer to every learning motivation statement, i.e. who identified with the statements stagnated or decreased in every control school and pilot school. While this is positive in the case of Meszes, at the opposite end of the scale, the number of students who did not identify with any of the statements also increased.
・ The output test found that the level of reading skills improved in every group. The most spectacular development occurred in Somogy, where the rate of finishing level students rose from 25% to 88%, albeit none of the students achieved the optimal level upon output.
・ In terms of maths, the rate of correct answers of the pilot students in Meszes and the Bártfa Street schools increased and the rate of incorrect and missing answers decreased. Conversely, in the Somogypilot group’s results
Table 18. Evaluation of maths tests by a teacher upon input.
Table 19. Evaluation of maths tests by a teacher upon output.
Table 20. Distribution of students by school.
deteriorated in respect of all three indicators.
Figure 9 and Figure 10 shows a significant positive change in the students’ self-image between the input and the output tests. The number of students making 8 - 9 positive statements increased in all three pilot schools. The biggest improvement is seen in the Meszes School pilot group. While in the control schools there was a 16% increase in 8 - 9 positive self-image statements upon output compared to the input stage, the same was 41% among the students participating in the CP programme.
3.2. Social Competence
In terms of social competence, the entire group displays a significant positive change between the input test and the output test. As shown in Figure 11 and Figure 12 the most marked change occurred in the Somogy School group, where none of the students gave 8 - 9 positive answers at the input and this rate was 38% upon output (an increase from zero to three students). In the control schools the rate of students giving the highest number of positive answers was up by 67%, and among pilot students, by 160%.
A different trend emerges when empathy is examined. As presented in Figure 13 and Figure 14 in this case the rate of children giving the maximum number of positive answers diminishes in all of the schools with the exception of the Bártfa Street institution. Somogy displays the biggest change: in the output test none of the students made four positive statements. The analysis revealed that the negative trend was stronger in the control schools, where there was a 40% drop in the number of students giving four positive answers in contrast to pilot schools, where there was an 11% increase thanks to Bártfa.
3.4. Learning Motivation
Figure 15 and Figure 16 show that learning motivation takes a shape similar to empathy. With the exception of the pilot group at Meszes School, the rate of student giving all-positive answers decreased in every group. Yet the picture in the Meszes pilot group is not entirely rosy as there was a significant increase in the number of those who gave few (0 - 4) positive answers-students from the middle range moved either upwards or downwards.
Figure 9. Self-image of students by school upon input.
Figure 10. Self-image of students by school upon output.
Figure 11. Social competence of students by school upon input.
Figure 12. Social competence of students by school upon output.
Figure 13. Empathy of students by school upon input.
Figure 14. Empathy of students by school upon output.
Figure 15. Learning motivation of students by school upon input.
Figure 16. Learning motivation of students by school upon output.
3.5. Reading Level
By the end of the programme reading levels improved in every institution, and more students had optimal and finishing reading levels than upon input. That change is presented in Figure 17 and Figure 18. While no student in Somogy achieved the optimal level even by time of the output test, the rate of finishing readers was up from 25% to 88%. In the control groups, the rate of students in the lowest (advanced) category dropped by 77% by the time of the output testing (from 9 to 2 students). The same decrease among pilot students was 70% volt (from 10 to 3 students).
3.6. Maths Results
The analysis of the mathematics component of the tests provides a mixed picture: there are groups in the control population as well as the pilot population where the result deteriorated while other groups performed better. Figure 19 and Figure 20 show that the pilot groups in Meszes and Bártfagave a higher number of correct answers to the mathematical problems and exercises in higher rates, while at the same time the rates of incorrect or missing answers decreased. On the other hand, in the Somogy School the pilot group’s results slipped in terms of all three indicators. It appears that the effect of the programme is manifested in different ways in different schools, communities and student groups.
Figure 17. Reading level of students by school upon input.
Figure 18. Reading level of students by school upon output.
Figure 19. Distribution of standardized average maths results by school upon input.
Figure 20. Distribution of standardized average maths results by school upon output.
4. Learning Motivation and Empathy through Highlighted Statements
Based on the findings presented so far it appears that while the participating students’ self-image has improved significantly and their social skills have also developed, their learning motivation has not changed and their empathy has not showed a positive development. This chapter addresses these two areas in greater depths.
We present two highlighted statements and explore to what extent the students identified with each at the input stage and at the output stage. The student population was again broken down in terms of financial conditions. The analysis revealed positive results in this area, too. In the least affluent group the number of students who were willing to learn things that fall outside their interest has increased. No unequivocally positive result was found in respect of the highlighted statement related to empathy but it is conspicuous that the number of affluent students who feel for their mates who are mocked at has not decreased.
4.1. “I Only Learn What I’m Interested in”―Learning Motivation
The first highlighted statement referred to learning motivation: students participating in the CP programme had to decide to what extent they can identify with the following statement: I only learn what I’m interested in. Figure 21 clearly shows that at the time of the output test the rate of those who felt the statement entirely described them decreased in all three categories. It is an important achievement that the attitude of student with the poorest background (those possessing 0 - 2 items) has completely changed: after the initial 27.8% only 16.7% identified fully with the statement. Maybe these students think they are not at all interested in maths and yet as a result of the programme they started paying attention in class.
Figure 22 examines the responses to the same statement but here the students were grouped according to the number of items possessed by the family. The findings are similar: the number of students fully identifying with the statement decreased in all the categories; and in the group where the family has the fewest possessions not only did the number of students fully identifying with the statement decreased but the number of those who felt the statement quite described them or only slightly described them also increased.
4.2. “If Someone Is Mocked at I Always Feel Very Sorry for Them”―Empathy
The second highlighted statement maps the students’ empathy. As was the case with the results presented in previous chapters of this report, here again the findings do not point in the same direction. Results are presented in Figure 23 and Figure 24. For instance in the group of students possessing 0 - 2 items the number of those who fully identify with the statement decreases, but the number of those who think that the statement is not at all typical of them likewise decreases. The movement is quite the opposite in the middle group (of students possessing 3 items) where the number of the most negative and most positive relies equally increased.
Figure 21. Students’ learning motivationanalysed through a highlighted statement (by items possessed by the student).
Figure 22. Students’ learning motivation analysed through a highlighted statement (by items possessed by the family).
Figure 23. Students’ empathy analysed through a highlighted statement (by items possessed by the student).
Figure 24 puts the same statement into focus but here the groups are established on the basis of the family’s possessions.
5. Summary: Findings of the Quantitative Analysis of the Pilot Mathematics Programme
Comparison of the results of the pilot and control groups revealed that in the period between the input and output
Figure 24. Students’ empathy analysed through a highlighted statement (by items possessed by the family).
tests the self-image of the students participating in the CP programme improved, as did their social competence, to an extent that exceeded the improvement of control group students. However, empathy and learning motivation did not improve significantly in the pilot group or the control group.
The analysis based on the students’ financial background highlighted the fact that the self-image of students in the lowest income categories did not improve as a result of the programme, so this positive outcome was not shared by this group. Students from affluent families typically show a lower level of empathy in both the pilot and the control group but in the pilot group the empathy level of students with poorer backgrounds also decreased, which may further widen the gap between the two groups. Participation in the CP programme had a positive effect on the learning motivation of children with a more solid financial background but made no appreciable impact on the motivation of the poorest students. On the other hand, it was a consequence of the programme that there was an increase in the number of students from poorer families who were willing to learn things other than what they were interested in. Nevertheless, on the whole, the better the student’s family background the more likely they are to give positive answers to attitudinal questions.
The improvement in the reading level of students in the pilot group was considerable and exceeded that of the control group. The improvement in their rate of correctly solved mathematical exercises and problems also exceeded that of the control group, and their mathematical confidence increased.
A source of the positive trend in the pilot group is that students from more affluent backgrounds gave significantly more correct answers in the output test than upon input, but is conspicuous that student with less fortunate backgrounds also tried harder and had greater confidence when faced with the maths test, even if they could not solve the problems correctly.
Analysis by institution highlighted the fact that the pilot students in the various schools moved mostly in the same direction in terms of the indicators, therefore it is likely that the programme has a consistent impact that is independent from the institution, i.e. the findings described above are valid to not just specific groups of students.
6. Limitations of the Study and Directions for Future Research
The research design of the study has been improved together with the pilot phases of the project. During the first pilot phase only the views and opinions of the students being members of the treatment groups were collected, the second phase introduced a control group and pre- and post-testing of the students. After the second pilot, Creative Partnerships Hungary project joined the OECD CERI project, called Assessing progression in creative and critical thinking skills in education, within which more than 800 students will be measured from grade 3 and grade 7. So the limit of this study, the small number of participant children, will be overcome, and this OECD project will enable us to check the validity of our results on a much bigger school population.
4The evaluation of the first pilot programme largely followed the internal evaluation mechanism of the original Creative Partnerships programme. This meant that the creative professionals prepared a so-called initial planning document which could then be compared to the data and information of the completion form they filledat the end of the programme. In addition a qualitative assessment took place in the final stage of the programme at each of the sites, which consisted of observation of sessions, semi-structured interviews with the school heads, structured interviews with the teachers and creative practitioners, and focus group sessions with the students (The qualitative study was carried out jointly by Creativity Culture and Education of England and T-Tudok). To supplement this internal practice and relying on the student questionnaire developed for the Complex Art Education component of the day school programme of the Hungarian Institute for Educational Research and Development, at the end of the programme T-Tudok sent the participating students an online questionnaire. In addition to the demographic data, the questionnaire collected the students’ observations about the programme and the school on the one hand, and attempted to explore the students’ self image, social competence, empathy and learning motivation by means of a test consisting of 50 items. The data captured in this way serve as a basis of comparison for the analysis of this year’s findings, see the quantitative analysis description in this report.
5About the survey in detail: http://negyedikesmeres.oh.gov.hu/#.
6The test was written by: Jánosi Amália (math teacher).
7k%p is the actual result achieved in relation to the optimal mastery of the basic skill and competency.
 Office for Standards in Education (2006). Creative Partnerships: Initiative and Impact. The Impact of Sustained Partnerships between Schools and Creative Practitioners as a Result of Creative Partnerships Initiatives. London: Office for Standards in Education.
 OKM (2012) Országos Jelentés a 4. évfolyamos Országos Készség-és Képességmérés Eredményeir?l (Ministry of Education and Culture Report on the Findings of the Nationwide Competencies and Skills Assessment in Grade 4). Budapest: Oktatási Hivatal [Hungarian Educational Authority].