The Faculty of Medicine of the University of Geneva considers it important that medical students, regardless of their ultimate career choice, be exposed to independent general practice. Consequently, the Faculty decided to incorporate a mandatory one-month rotation during clerkship in a primary care physician’s private office (OFSP, 2012). The ambulatory setting offers a privileged learning environment that allows both the observation and the follow-up of acute and chronic illnesses. It also fosters learning of preventive medicine, medical interview communication techniques, and the management of the many psychosocial dimensions of illness (Dent, 2005; Irby, 1995; Ramani & Leinster, 2008). The expected benefits of such a rotation include the personalized access to a population of patients for a sustained period of time and the development of a mentoring relationship with an experienced physician.
The primary care outpatient ambulatory setting represents a considerable challenge as a learning environment for the supervisor and the student. This is because, typically, supervisors have little time for planned and structured teaching, interactions between students and physicians are brief, and supervisors have little or no control over the organization of the working hours, the flow of patients, and the clinical situations (Ramani & Leinster, 2008). We were interested in delving deeper into this specific reality to uncover areas where clinical supervisors might be provided with greater support.
A preliminary study carried out in Geneva (Muller-Juge et al., 2018) showed that primary care physicians have strong motivation to welcome and integrate students in their work environment. This is mainly because they wished to promote primary care practice, and because they enjoyed being the “witnesses” to the “first times” of the budding physicians (the first sutures, first case of pneumonia, etc.). In this perspective, these primary care physicians consider themselves role models and mentors to their students. At the same time, it has quickly become apparent that a more structured approach was required for the supervision of students in such a clinical setting. Muller-Juge et al. (2018) highlighted that these physicians faced significant challenges when they had to adapt to the student’s presence in their workspace. Supervisors didn’t anticipate the cognitive load required to explain their clinical reasoning processes, the supervision planning, and the emotional load not only of having to “share” their patients with a student, but also to take on the responsibility of their emotional reactions (Muller-Juge et al., 2018). It also appeared that these primary care physicians were surprised to find out that it was a challenge to conjugate their two roles: the role of a clinician who takes care of patients, and the role of teacher (Audétat, Grégoire, Fernandez, & Laurin, 2017).
This situation led us to consider developing and delivering a targeted longitudinal Faculty Development program for these primary care physicians. This program includes an initial three-hour workshop required to supervise and welcome students in their practice. They are then invited to targeted workshops and required to participate in a yearly half-day continuous training organised by the primary care unit (UIGP) of the Faculty of Medicine (McLean, Cilliers, & van Wyk, 2010; Steinert & Mann, 2006). During this half-day, aimed at developing a community of practice and to foster the ability to transfer in their context, participants learn needed competencies, as peer-teachers guide them through relevant content, demonstrations, practices and feedback (Yelon, Ford, & Anderson, 2014).
1.1. Our Conceptual Framework
Stalmeijer’s (Stalmeijer et al., 2013) work on clinical supervision focuses on the teacher’s actions to create a safe learning environment, in which students feel free to ask questions and are not under pressure if they fail or make a mistake. The quality of these interactions between supervisor and supervisee should allow feedback and observations, both offered and received in a constructive manner, and promote students’ development of autonomy and reflexive thinking. Clinical teachers are also expected to demonstrate relevant skills, to be good role models, and to develop their thought process out loud, explaining their clinical reasoning. Following this model, supervisors will be encouraged once they feel confident enough to delegate more tasks to the student. Hence, cognitive apprenticeship describes a continuum from shadowing (the learner follows the teacher everywhere, has little to do but observe the teacher’s demonstrations and explanations, to fading (the teacher gradually recognizes the student’s autonomy and progressively reduces support, i.e. a scaffolding, until the end of the training). By doing so, it would allow supervisors to explore and identify some learning gaps regarding specific areas or practice.
Out of this body of work, Stalmeijer and her colleagues developed the five dimension model—the Masstricht Clinical Teaching Questionnaire (MCTQ)—to assess the quality of the clinical supervision, that is role modeling, coaching, articulating, exploration, and learning environment (Stalmeijer, Dolmans, Wolfhagen, & Scherpbier, 2009). Role modeling is embodied by teachers when they actively demonstrate and describe their reasoning while they are working. Coaching refers to teachers observing students and providing specific and concrete feedback on their performance. Articulation involves teachers questioning students (for example about their clinical reasoning) and encouraging them to ask questions. Exploration is aimed at encouraging students to formulate and pursue personal learning goals. Learning environment is about setting a constructive learning climate, which clinical teachers can foster by showing an interest in students’ learning and making students feel respected. All five dimensions require different pedagogical competencies.
1.2. Aim of Our Study
The learning needs’ assessment regarding clinical supervision and the necessity to develop strategies that foster competency development in a complex environment such as primary care outpatient ambulatory settings constitutes our main goal (O’Sullivan & Irby, 2011). As shown above, there are tensions surrounding the dual professional identity that need to be resolved and there is a clear need to develop mentoring relationships with clerks who are inexperienced and who might not be aiming to practice primary care. While clinicians are glad to welcome students in their office, the complexity inherent in the rotation make the task of supervising clerks very arduous. Thus, a more nuanced portrayal of the situation, gleaned from multiple perspectives, might allow us to identify relevant training priorities. Hence, perceptions of the components of the 5 dimensions of the MCTQ held by students and their supervisors were contrasted so that the differences might shed light into specific training needs. To the best of our knowledge, there is little research that has focused on an assessment based on students’ and teachers’ dual perceptions of the quality of clinical supervision.
Hence, our research question was to ascertain whether the supervisors’ self-assessment coincide with the learners’ assessment regarding the behaviors associated with the five dimensions. On which dimensions do perceptions converge and on which do they diverge?
This study was a prospective observational study based on self-administered questionnaires filled out by students and the clinical supervisors.
We chose the Maastricht Clinical Teaching Questionnaire (MCTQ) developed by Stalmeijer et al., which is a valid and reliable evaluation instrument (Stalmeijer, Dolmans, Wolfhagen, Muijtjens, & Scherpbier, 2010). It has been also used as a self-assessment instrument for veterinary clinical teachers (Boerboom et al., 2011), and for clinical supervisors from acute and community settings in Australia (Bearman et al., 2018).
Figure 1 presents the MCTQ for the students, which has 14 items grouped in five dimensions. The questionnaire was reworded so as to make it suitable as a self-evaluation tool for the clinical teachers. Both questionnaires consist of essentially the same questions, only the wording was modified from “My clinical teacher …” to “I …” For example the item “My clinical teacher showed that he or she respected me” was turned into “I feel able to show respect for my student.”
Figure 1. Questionnaire items.
in the self-evaluation questionnaire. Each item is assessed with 5-point Likert scale where 1 = fully disagree, 5 = fully agree).
We were aware of the limitations of self-reported data, in particular that they may be widely divergent from real practice (Norman, Norcini, & Bordage, 2014). However, self-reported data indicate how clinical supervisors themselves identify “on the ground” areas for improvement. In this perspective, it is more closely aligned with a pedagogic strategy of self-reporting data rather than the ability to self-assess (Eva & Regehr, 2008).
All students at the University of Geneva Faculty of Medicine are required to enroll in a one-month mandatory rotation in outpatient primary care. During this rotation, they are paired up with a physician working in a private practice, who is responsible for supervising their work.
We conducted this research during the academic years (autumn to summer term) of 2016-17 and 2017-18. Students (N = 275 for the two years) were asked to fill in the questionnaire once they had completed their rotation. Supervisors (N = 128) were asked to fill in the questionnaire via Qualtrix at the end of the rotation.
As a first step summary statistics (given as mean were computed ± standard deviation) for every dimension for the students’ questionnaire, respectively the supervisors’ questionnaire. Should one item be missing, we just took the mean of the available items of the same dimension for the evaluation of the mean. We then investigated, using t-tests and for every dimension, the potential differences between the mean of the evaluations by the students and the auto-evaluations.
The basic analyses mentioned above do not take a potential supervisor effect, i.e. for a given supervisor the evaluations and auto-evaluation may be very similar, whereas for another the evaluations could be much higher than the auto-evaluation. Since there were potentially several evaluations available for every supervisor, we thus investigated the link between the evaluations and the auto-evaluation using a linear-mixed effects model, taking the evaluation as the dependent variable, the auto-evaluation as the independent variable, and incorporating a supervisor random effect (on both the intercept and the slope). A perfect theoretical adequacy between them would thus lead to a regression of slope one and intercept zero, and a minimal between-supervisor variability. Sub analyses, using the same methodology, were also done for each of the five dimensions.
All the analyses were performed with R version 3.3.3 (The R Foundation for Statistical Computing, Vienna, Austria).
A total 275 questionnaire completed by the students and 50 completed by the supervisors were used in the analysis. Supervisors may have supervised more than one student, so that 50 auto-evaluations were completed by 33 supervisors.
3.1. Evaluations and Auto-Evaluations
Table 1 presents the averages (± standard deviation) for each of the five dimensions of the Stalmeijer model obtained from the 275 students and the 33 supervisors.
All average scores were higher than 3, with the higher scores observed for the climate, and the lower for the clinical reasoning. Apart from coaching, the evaluations made by the students scored higher than the self-evaluation by supervisors.
3.2. Association between Evaluations and Auto-Evaluations
The linear mixed-effects model showed an average evaluation of 3.54 ± 0.45 (p < 0.0001) with no evidence of a link with the auto-evaluation (p = 0.882), see Figure 2. The evidence of a significant difference between-supervisor variability was weak (p = 0.054; 9.1% of variance explained).
Results were similar for all five dimensions of Stalmeijer’s model.
Both students and supervisors answered the Maastricht Clinical Teaching Questionnaire. The results clearly show that students consistently scored their supervisor higher on all five dimensions than the supervisors did themselves. Several reasons may explain these findings: First, the difference in scores may be indicative
Table 1. Summary statistics regarding the five dimension of the MCTQ.
Figure 2. Scatterplot of the association between the auto-evaluations (x-axis) and the evaluations by the students (y-axis). The light-blue line was obtained by smooth-spline.
that trainees and supervisors need to take more time to discuss and express each other’s needs and expectations for the rotation. It seems to be still rare for supervisors to make explicit what they expect from their students. Second, students are quite used to judging their teachers and supervisors in the academic context. They regularly fill in forms on which they express their satisfaction (or not) with different aspects of the teaching and learning experience. The context of the private practice is somehow different: clinical supervisors in private practice are not used to be assessed, neither to assess themselves, and as our results show, do so more severely than students.
The results are such that we do not really know whether the rotation went as well as the students said or as badly as the supervisors said. This discrepancy may point to a lack of reflexive habit of exploring one’s teaching performance. In this sense, our results suggest that we need to reinforce teachers’ reflexive practice in order to have a more balanced view of their teaching practice.
It is interesting to note that the students gave their highest scores on the dimension of learning environment, and the dimension of role modeling, (see Table 1). As did the clinical teachers: This is congruent with the Bearman’s results (Bearman et al., 2018). It is satisfying that both students and clinical teachers recognized the significance of the relationship in the clinical teaching. Kilminster, Cottrell, Grant, and Jolly (2007) identified these type of skills or qualities as contributing to effective supervision. Teachers gave their lowest scores on the articulation dimension while students assessed it higher. This leads us to hypothesize that the support for the development of clinical reasoning, at the heart of the medical profession, could be improved. These results could be interpreted as a call to improve skills in intervening and supporting students as they struggle to acquire clinical reasoning skills. These skills would support adoption of a professional approach to teaching rather than the intuitive approach that appears to be reflected in these findings as well as in other results (Atkinson, Ajjawi, & Cooling, 2011; Audétat, Laurin, Dory, Charlin, & Nendaz, 2017).
It is also worth reporting that both supervisors and students score themselves low on the exploration dimension. Encouraging students to formulate and pursue personal learning goals is often overlooked; these findings are congruent with other research highlighting that supervisors tend to fail to formally conclude the supervision and struggle to explain to the student what has been learned and what remains to be learned (Pereira Miozzari, Rieder, Sader, Sommer, & Audétat, 2020 ).
Finally, students gave a low score on the coaching dimension, while clinical teachers gave themselves a good score on the same dimension. These findings confirm previous results showing that teachers usually believe they provide regular and sufficient feedback, but this is not how it is perceived by learners (Cantillon & Sargeant, 2008; Liberman, Liberman, Steinert, McLeod, & Meterissian, 2005). Providing specific and concrete feedback is the cornerstone of effective clinical teaching (Hesketh & Laidlaw, 2002). Clinical teachers should consider feedback as a crucial skill to be acquired through repeated practice and augmented by reflection on their own performance (Cantillon & Sargeant, 2008). Such critical skills could also be the subject of focused training or coaching sessions.
Strengths and Weaknesses of Our Study
The major limitation of the study is the relatively low number of self-assessments: 33 out of 128. The questionnaire was disseminated via institutional e-mail and there was no obligation for the clinical teachers to fill it out. Another approach should be sought in which more incentive is provided to the supervisors to take the time to fill out the questionnaire.
The results of our study point to some improvements to the training of clinical supervisors. Although it appears that students and supervisors do focus on the same issues pertaining to clinical supervision, and student expectations might not coincide with supervisors’, the fact remains that they experience the same tasks described in each of the five dimensions of the model. The divergence of opinion contains many insights into what happens and more importantly on what could be improved upon.
 Atkinson, K., Ajjawi, R., & Cooling, N. (2011). Promoting Clinical Reasoning in General Practice Trainees: Role of the Clinical Teacher. The Clinical Teacher, 8, 176-180.
 Audétat, M.-C., Grégoire, G., Fernandez, N., & Laurin, S. (2017). From Intuition to Professionalization: A Qualitative Study about the Development of Teacher Identity in Internal Medicine Senior Residents. International Journal of Education and Health, 1, 47-53.
 Audétat, M.-C., Laurin, S., Dory, V., Charlin, B., & Nendaz, M. (2017). Diagnosis and Management of Clinical Reasoning Difficulties: Part I. Clinical Reasoning Supervision and Educational Diagnosis. Medical Teacher, 39, 792-796.
 Bearman, M., Tai, J., Kent, F., Edouard, V., Nestel, D., & Molloy, E. (2018). What Should We Teach the Teachers? Identifying the Learning Priorities of Clinical Supervisors. Advances in Health Sciences Education, 23, 29-41.
 Boerboom, T. B. B., Dolmans, D. H. J. M., Jaarsma, A. D. C., Muijtjens, A. M. M., Van Beukelen, P., & Scherpbier, A. J. J. A. (2011). Exploring the Validity and Reliability of a Questionnaire for Evaluating Veterinary Clinical Teachers’ Supervisory Skills during Clinical Rotations. Medical Teacher, 33, e84-e91.
 Dent, J. A. (2005). AMEE Guide No 26: Clinical Teaching in Ambulatory Care Settings: Making the Most of Learning Opportunities with Outpatients. Medical Teacher, 27, 302-315.
 Eva, K., & Regehr, G. (2008). I’ll Never Play Professional Football and Other Fallacies of Self-Assessment. Journal of Continuing Education in the Health Professions, 28, 14-19.
 Kilminster, S., Cottrell, D., Grant, J., & Jolly, B. (2007). Effective Educational and Clinical Supervision: AMEE Guide No 27. Medical Teacher, 29, 2-19.
 Liberman, A., Liberman, M., Steinert, Y., McLeod, P., & Meterissian, S. (2005). Surgery Residents and Attending Surgeons Have Different Perceptions of Feedback. Medical Teacher, 27, 470-477.
 McLean, M., Cilliers, F., & van Wyk, J. (2010). Faculty Development: Yesterday, Today and Tomorrow, AMEE Guide in Medical Education No. 33. Dundee: Association for Medical Education in Europe (AMEE).
 Muller-Juge, V., Pereira Miozzari, A. C., Rieder, A., Hasselga°rd-Rowe, J., Sommer, J., & Audétat, M.-C. (2018). A Medical Student in Private Practice for a 1-Month Clerkship: A Qualitative Exploration of the Challenges for Primary Care Clinical Teachers. Advances in Medical Education and Practice, 9, 17-26.
 OFSP (2012). Office fédéral de la santé publique: Masterplan Primary Care [Faktenblatt, Masterplan Hausarztmedizin und medizinische Grundversorgung].
 Pereira Miozzari, A.-C., Rieder, A., Sader, J., Sommer, J., & MC, A. (2020). Clinicians or Teachers, Why Not Both? Expanding the Comfort Zone of Swiss Clinical Teachers in Private Practice.
 Stalmeijer, R., Dolmans, D. H. J. M., Snellen-Balendong, H. A. M., Van Santen-Hoeuff, M., Wolfhagen, I. H. A. P., & Scherpbier, A. J. J. A. (2013). Clinical Teaching Based on Principles of Cognitive apprenticeship: Views of Experienced Clinical Teachers. Academic Medicine, 88, 1-5.
 Stalmeijer, R., Dolmans, D. H. J. M., Wolfhagen, I. H. A. P., Muijtjens, A. M. M., & Scherpbier, A. J. J. A. (2010). The Maastricht Clinical Teaching Questionnaire (MCTQ) as a Valid and Reliable Instrument for the Evaluation of Clinical Teachers. Academic Medicine, 85, 1732-1738.
 Stalmeijer, R., Dolmans, D. H. J. M., Wolfhagen, I., & Scherpbier, A. (2009). Cognitive Apprenticeship in Clinical Practice: Can It Stimulate Learning in the Opinion of Students? Advances in Health Sciences Education, 14, 535-546.
 Yelon, S. L., Ford, J. K., & Anderson, W. A. (2014). Twelve Tips for Increasing Transfer of Training from Faculty Development Programs. Medical Teacher, 36, 945-950.