Received 26 April 2016; accepted 10 June 2016; published 13 June 2016
Since 2005, several efforts have been made in India through the National Rural Health Mission (now called National Health Mission―NHM) to increase institutional deliveries   . Yet despite these effort, India, still lags in achieving the 2015 Millennium Development targets for maternal and child health   . Quality of care is often cited as a key impediment to improving these indices and is known to be impacted by several factors such as individual health care providers (HCPs) knowledge and skills; systems level constraints such as availability of infrastructure, equipment, supplies; support systems viz. management, information systems, resources, accountability  -  , inadequate systems to monitor, poor regulation of pre-service educational institutions, professional practice and clinical standards  . Other additional reasons for poor quality of care delivery include lack of consistent structured education at pre-service and during service, lack of access to evidence based current information, lack of written learning packages and protocols to acquire and evaluate skills   .
Nurses, the frontline health care professional of primary health centers (PHC) in India are mostly qualified through general nurse midwife 3.5-year diploma program. They are expected to primarily offer skilled birth care, including screening, initial management and referral of maternal and newborn complications to higher level facilities called first referral units. Major knowledge and skills gaps in provision of maternal and newborn care across public and private health facilities were identified through a project done in northern Karnataka districts Sukshema   , with the most severe gaps being amongst nurses and auxiliary nurse midwives. This is despite the attempts made by the Government of India to upgrade their competencies through class room based continuing education (CE) programmes such as the 2 - 3 weeks duration of skilled birth attendant (SBA)  and 2-day essential care of newborn (Navjot Shishu Suraksha Karyakram―NSSK  ). It thus makes one question the value of one time off-the-job training as an effective CE for improving competencies. It also throws light on the need to test for large scale implementation, CE programme innovations  -  for nurses so that they can facilitate better outcomes and provide safer and quality care.
Until recently, CE was not a mandatory requirement in India for an active nurse midwife registration   . Yet standards for credit hours have been developed by only a handful of state nursing councils. Given that knowledge gained through basic education has a half-life of 2.5 years  , it is essential that innovative CE methods are created, tested and implemented so that they are timely, regular, accessible and relevant to the health care practice and to the practitioner  . One of these innovations could include onsite mentoring―a method that has not been tried in the public health system in India, on large scale but reported as beneficial for professional growth and competency development amongst novices in the health field, with presumable robust gains to the whole health care delivery system   . Mentoring has neither caught up as a concept in the public health system, and it was thus postulated that on-site mentoring would contribute to improved health worker performance and health care quality (including referral management) at PHCs  . There is paucity of information on what a good onsite nurse mentor program could entail in order to impact on intranatal and early postnatal/newborn quality of care in PHC settings. Considering the ground realities of rural and remote northern Karnataka, the project team conceptualised a program to train a group of registered nurses to mentor HCPs in the PHCs. This paper only describes the process evaluation of a nurse mentor training programme, while details of the impact of nurse mentors in the field can be obtained from another article  .
2.1. Mentor Training Package and Other Intervention Detail
Specialists from the field obstetrics, newborn care, public health, quality improvement, training and epidemiology did an extensive review of literature, brainstormed on expected roles  , and came to a consensus on required competencies of such a cadre of nurse mentors. Inputs known to be crucial for establishing strong and effective mentoring relationships were considered in this design. These included conducting reasonably intensive screening of potential mentors as described above; providing more than six hours of training; offering post- match training and support  , systematic monitoring of program implementation, structured format with established frequency for mentoring, on-going training for mentors and the support for mentors in the field   . The menu of suggested methods for assessing mentors such as direct observation, use of objective structured clinical evaluation in simulated settings, testimony of others and self and peer assessment   were also key considerations. The training package was designed using Government of India (GoI)   and where not available the World Health Organisation (WHO) guidelines  .
2.2. Ethical Considerations
Permission was first obtained from the Department of Health and Family Welfare-Karnataka government and the Institutional ethics committee of the institution. Oral informed consent was obtained from the nurse mentors, prior to collecting any information.
2.3. Recruitment and Selection of Nurse Mentors
Recruitment of nurse mentors was done through open advertisements. They were selected using key selection criteria such as being located in the district, having an active nurse midwife registration, educational qualifications (masters degree desirable), aptitude, clinical-teaching-managerial experience(desirable), and willingness to travel; and finally by a written test and interview. All nurse mentors who consented, were available for the assessment were included for the process evaluation.
Intervention consisted of the induction training, clinical posting, onsite supportive supervision visits and intermittent refresher training and district clinical posting.
1) The induction training was a 5 week interactive participatory package with tailor made content on obstetric- intrapartum, early postnatal and basic newborn care; mentoring, quality improvement (QI), and systems issues such as infection control, documentation, referrals and supplies (Table 1). Since the documentation system at the PHC did not permit proper audit of care provided, new case sheets that included a delivery record and complication case sheets were developed for the purpose of the project. The case sheets were like job aids that facilitated decision making, and had boxes to tick for actions taken. They were based on guidelines    available on essential assessment of a woman in labour; partograph; active management of third stage of labour; routine fourth stage of labour, early postnatal and neonatal care; identification of complications with their initial management and actions before referral. Nurse mentors were oriented to its use. The complications case sheets included preterm labour, antepartum haemorrhage, hypertensive disorders of pregnancy, prolonged labour, postpartum haemorrhage, sepsis and a newborn case sheet that addressed low birth weight care and sepsis.
Table 1. Organisation of training package for nurse mentors on essential care during labour, delivery and early postnatal and newborn period.
2) Participants were given the SBA  , NSSK  and training manuals  -  for their reference. A log book was provided to mentors to monitor procedures they completed under supervision or if they observed these during their clinical postings. They were also asked to practice using case sheets, so that they would become confident in its use before being deputed to the field. Attempt was made to see that groups of 4 - 5 nurse mentors were directly mentored in the clinical area by a specialist nurse or doctor. Daily debrief of theory or clinical sessions and extra time to practice on mannequins was done to enhance learning by sharing, seeing and doing.
3) Additional clinical posting and structured mentoring plans: To build on their competencies and confidence, additional clinical postings for a week in a hospital similar to a PHC were planned soon after the induction training for nurse mentors in batches of 5 - 6.
Structured plans for mentoring were given to them before they assumed the responsibilities in their respective districts.
4) Onsite supportive supervision visits by obstetric and newborn specialists (doctors or nurses) from the tertiary level teaching hospital (located 300 - 500 km away from the districts), were organised once in 3 - 4 months to support nurse mentors in the field. The specialists had a 1-day standardisation training on guidelines of care    and the project, a 1-day orientation on clinical mentoring and their roles, access to guidelines and a structured format to follow during their visit, to ensure uniformity of messages to nurse mentors.
Two to three PHCs were visited during these 2 day visits by specialists. In all 28 visits were made to the 8 districts during the 14 month duration. Direct observation of practice, supplies, infection control measures and case sheet audits were performed. A participatory debrief reinforcing positive points and sensitisation on gaps was done with all the mentors and the program specialist of the district. A report of the visit provided feedback and inputs for specialists to use at the next visit.
5) Intermittent refresher training and clinical practice in the district hospitals were planned for nurse mentors in their own districts to strengthen and sustain their clinical competencies
Nurse mentors were recalled for a last 3 day refresher 12 - 18 months after their induction training to the teaching hospital. Difficult case situations faced in the field were gathered from nurse mentors and used for sessions.
Evaluation of the impact of nurse mentoring training program could be inferred from, changes in knowledge and competencies of cadre of nurse mentors from different timelines. In this paper the process evaluation of the nurse mentor training program using an evaluation framework (Figure 1) is described under baseline, immediate and long term assessment of clinical knowledge and competencies (Figure 2):
Figure 1. Process evaluation framework of nurse mentor training program.
Figure 2. Timeline of interventions and assessment.
1) Baseline assessment: Profile of nurse mentors and 50 item validated (T0) knowledge questionnaire was administered on the first day of the training. The T0 consisted of either completion or multiple choice items using the problem based approach. There were 12 items on mentoring and QI content (not presented in this paper); 18 on obstetric content and 20 on newborn content. An answer key was available for scoring participants based on their responses. Items on obstetric content included calculation of expected date of delivery or gestational age, focus on actions to be taken during the different stages of labour, identification of complications such as hypertensive disorders of pregnancy, postpartum haemorrhage, sepsis and prolonged labour. Items on basic newborn care focused on routine care, breast feeding, thermal control especially for low birth weight babies, identification of complications-sepsis, basic newborn resuscitation.
2) Immediate assessment: On the last day of the training, knowledge test was repeated (T1) and competencies were assessed through objective structured clinical examination (OSCE1). Ten stations, five observed and five unobserved were designed for both obstetric and newborn content respectively based on key competencies. At unobserved stations, participants were expected to provide answers to a question on specific content. Answer keys were available for scoring responses. The observed obstetric stations included measurement of blood pressure, abdominal examination, postnatal counselling, per vaginal examination and active management of third stage of labour. The unobserved stations were calculation of gestational age, partograph use based on a case scenario, preparation of labour room, complication case sheet filling for postpartum haemorrhage, and medication for severe pre-eclampsia. The observed stations for newborn care included bag and mask resuscitation, checking weight, counselling for referral and discharge of low birth weight, mentoring a nurse checking newborn temperature. Unobserved stations included use of radiant warmer, danger signs of newborn, infection control measures, signs of good attachment and position during breast feeding and case sheet entry.
3) Long-term assessment of knowledge was done just before their final refresher training (T2) using the same test and through OSCE 2 which was shorted to eight key competency stations, four each in obstetric and newborn area with two observed and two unobserved, for logistic limitations. The two OSCEs (1 and 2) were found to be equivalent  (p < 0.01). Both were conducted under the same conditions. A distinct advantage of this method of testing is it reduces testing threat to internal validity   . The time interval between the two OSCEs ranged from 11 - 19 (13.9 ± 2.8) months.
All knowledge and competency scores are expressed in mean percentages with standard deviation and confidence intervals (CI) unless the statistic was not normally distributed. Attempt was made to look at prior participation in government SBA training in Obstetric training and NSSK training and its impact on nurse mentors performance at baseline and acquiring new knowledge. Box and whisker plot is shown to depict minimum, first quartile, median, third quartile and maximum scores at baseline, immediate and long-term assessments for both knowledge and competency scores (Figure 3 and Figure 4).
Figure 3. Box and whisker plot showing baseline (T0), immediate (T1) and long-term (T2) knowledge scores of nurse mentors on obstetric (OB) and newborn (NB) content areas. PS: T0 to T1 OB and NB there is significant increase using Wilcoxson signed rank test (p < 0.0001); T1 to T2 OB significant increase using Wilcoxson signed rank test (p < 0.05) but no significant change in NB score (p > 0.05).
Figure 4. Box and whisker plot showing immediate (OSCE1) and long-term (OSCE2) competency scores of nurse mentors on obstetric (OB) and newborn (NB) content areas. PS: OSCE1 to OSCE2 OB and NB there is significant increase (p < 0.001) using Wilcoxson signed rank test.
3.1. Baseline Profile
In all 55 nurse mentors went through the induction training (Table 1). Four batches of 14 - 20 participants were trained over a period six months (August 2012 till January 2013). Almost all nurse mentors recruited had diploma in nursing training 52 (94.6%), 2 (3.6%) had degree in nursing training and 1 (1.8%) had a master’s degree in nursing training. A diploma nurse receives at least 100 hours of theory in obstetric and newborn care and nearly 800 hours of clinical hours; while a degree holder 90 hours and 360 hours respectively  during their basic nursing. While a master’s degree holder with obstetric specialisation would receive 300 hours theory and 1300 hours of clinical posting. The nurse mentors recruited for this project had limited work experience (median 24 mons, range from 8 - 144 mons) prior to the training. Only 5 (9.1%) reported to have had SBA training and 4 (7.2%) had NSSK training.
The induction training was for 5 weeks with a proportionate distribution of obstetric and newborn theory and practical sessions (Table 1). Mentors went through 98 hours of theory class room sessions and 105.7 ± 13.7 clinical posting hours. All were posted for one more week of clinical posting (118 hrs) in settings similar to PHCs before they assumed the mentoring responsibilities in their respective districts.
Nurse mentors reported exposure to 1 - 7 (3.3 ± 1.9) handholding visits by specialists from the teaching hospital in a year, 1 - 2 (1.5 ± 0.9) mentoring visits for infection control practices in the PHCs, 1 - 4 (2.5 ± 0.8) weeks of clinical postings and 1 - 4 (2.1 ± 1.2) refresher trainings in their respective district.
3.3. Baseline Assessment―Knowledge (T0)
Baseline scores expressed in percentage of nurse mentors was much below average (<25%) in newborn content areas (18.2 ± 19.1; antilog 4.02 - 32.8) but average for obstetric content (44.3 ± 12.7, CI: 38.8 - 49.7). Mann Whitney U test was computed to determine if baseline variables had an effect on T0 knowledge (Table 2). Mentors with prior SBA training had significantly higher knowledge scores (62.9%) in Obstetric content area (p = 0.02) compared to those without (42.3%). Those with prior NSSK training had significantly higher scores compared to those without it (p < 0.001), in obstetric (81.6% versus 41.2%), newborn (54.4% versus 14.9%) and overall content (53.4% versus 25.8%) respectively.
3.4. Immediate Assessment―Knowledge (T1) and Competency Scores (OSCE1)
Immediately after the induction training knowledge scores increased from baseline in both newborn content (66.4 ± 14.9; CI 61.6-71.2) than in obstetric (72.1 ± 13.8; CI 68.4-75.2). Wilcoxson signed-rank test was applied to test if the difference in scores from baseline to immediate assessment was statistically significant, this difference was statistically significant at p < 0.0001 (Figure 3). Baseline variable such as prior SBA or work experience did not impact on the change in score from T0 to T1 (Table 3). But knowledge scores of those with prior NSSK training were significantly lower than those without it (Obstetric 5.2% versus 30%; Newborn content 42.1% versus 50.7%) at p < 0.01.
Table 2. Association between prior training and baseline knowledge scores.
#Mann-Whitney U-test; *statistically significant.
Table 3. Association between prior training and change in knowledge scores from baseline (T0) to immediate assessment (T1).
#Mann-Whitney U-test; *statistically significant.
Competency scores as measured by OSCE1 were as follows: Obstetric (62.2 ± 13.2; CI 58.7 - 65.7) and newborn (52.6 ± 9.3; CI 50.2 - 55.1) content. There was a significant positive correlation between T1 and OSCE1 scores in both obstetric (r = 0.453 p < 0.01) and newborn (r = 0.571 p < 0.01).
3.5. Long-Term Assessment―Knowledge (T2) and Competency Scores (OSCE2)
When re-tested, just before the last refresher training, after approximately 13 months, there was an increase in their knowledge scores for obstetric content area (80.8 ± 11.4; CI 77.2 - 84.5) and in newborn content (66.4 ± 18.3; CI 61.6 - 71.2). Wilcoxson signed rank test showed a significant increase in obstetric scores (p < 0.05) but no significant change for newborn scores (p > 0.05) was noted (Figure 3).
The competency score increased from OSCE 1 to OSCE 2 in both obstetric content (69.6 ± 12.5; CI 66.7 - 72.5) and newborn content (63.5 ± 14.4; CI 58.9 - 68.2). This increase was significant for both obstetric and newborn scores (p < 0.001) when applying the Wilconxson signed rank test (Figure 4). Intervention such as refresher training, handholding visits, and clinical postings were not significantly correlated with T2 nor OSCE2 scores. There was a negative correlation between time period from induction training to and T2 and OSCE2 indicating a need to have a cut off line for refresher training, to facilitate retention of knowledge and competencies.
4.1. Profile of Nurse Mentors and Impact on Baseline Knowledge (T0)
A mentor is usually described as an older, experienced colleague with a nurturing personality who supports the inexperienced, younger nurses in the early months on the job. However in this project despite the intention to have nurse mentors who had at least 5 years of work experience, with a degree or masters in nursing qualification, this was not the case. Plausible reasons for this could include the fact that since extensive travel was anticipated, it was planned to recruit nurse mentors locally, the concept by itself was new for nurses in India, that the demands of the job might not have appealed to more experienced nurses with families and that the job did not offer long term security, since it was project based. Almost all nurse mentors selected for the project were with a diploma nurse qualification, had work experience mostly less than 2 years and little or no exposure to teaching.
From the findings at baseline assessment it was clear that prior SBA training did not affect knowledge scores. But prior NSSK training seemed to impact these scores of mentors (p < 0.05). It is thus important to keep in mind that prior SBA or NSSK training might not be essential criterion for selection of nurse mentors who would work onsite to improve quality of intranatal, early postnatal and neonatal care at PHCs.
4.2. Immediate Knowledge (T1) and Competency Scores (OSCE1)
The inputs on mentoring, QI and systems strengthening, obstetric and newborn content were closely intertwined in this training program through simulation, discussion and practice―all experiential learning methods and by specialists who were oriented to the objectives of the project. This could have accounted for threefold increase in scores immediately after training from baseline. The competency scores were significantly correlated to immediate knowledge post induction training (T1) scores thus confirming internalisation of concepts. The fact that all specialists involved in training and supportive supervision visits went through a standardisation and mentoring workshop, possibly helped in percolating the same messages, thus resulting in internalisation of the concepts by nurse mentors  .
Prior SBA training did not impact on the gain in knowledge from baseline and immediately after training (T0 to T1). Surprisingly, those with prior NSSK training had a significant lower gain in knowledge compared to those without it. We did presume that prior SBA and NSSK training would be assets to knowledge levels but this did not seem to be the case, and thus may not be essential criterion for change in knowledge levels. Positive points about the nurse mentors training as reported by them in their evaluation post the training program included the following “safe and good training for future life”, “I have confidence for conducting delivery and newborn care”, “During our mentoring visit we can use the basic skill that we have learned during the training”, “such kind of training must be conducted more frequently”, “mentoring role plays”, “on resuscitation and breast feeding…because we all were involved more in practical and also it is a new practice and easy method”. The experiential learning methods used both in the class room and clinical posting sessions did facilitate internalisation of these concepts, despite challenges of limited work experience and lack of SBA/NSSK training.
There was no significant correlation found between theory and clinical hours with T1 and OSCE1 scores respectively. Cognisance of balance between theory, class room practice and clinical hours is essential when designing a CE program to build competencies of mentors for quality improvement  -  in intranatal, early postnatal and neonatal care. In fact these novice nurse mentors indicated that they “required time to reflect and discuss their observations in the clinical area, for better learning”.
4.3. Long-Term Knowledge (T2) and Competency Scores (OSCE2)
Long-term knowledge and competency scores in obstetric and newborn content areas either increased or were sustained over a year. Typically this cadre of nurse mentors were supported for sustained learning through consciously planned structured visits for mentoring and by onsite field support through specialists visits; timely clinical postings in district hospitals and regular focused refresher training programs. Literature shows mentors must have a clearly defined purpose, require management support and commitment which is visible, involve training, involve off-line relationships, and have a strong monitoring and evaluation focus  -   -    . All of these were inbuilt into the follow up of this nurse mentor cadre.
Further mentors who were not confident or judged as not being competent would need more support. In this project too, mentors who were weaker (based on OSCE scores) had more exposure to specialist supportive supervision visit. Refresher training programs organised by the training institute must be spaced for not more than a year to ensure learning is sustained, as seen by lower T2 and OSCE2 scores among those who came much later for the final refresher compared to those who came earlier on. Confidence in competencies is key to success in mentoring relationships and thus efforts must be made to sustain these to expert level.
4.4. Lessons Learned
From this project it is clear that novice registered nurses with little or no experience can be moulded to take on the role of mentoring provided several conditions are met. Although at the start, they lacked confidence in basic obstetric and newborn care competencies, their clinical knowledge and competency scores improved after training and was sustained over a period of one year. This could possibly be associated with the on-site support they received from specialists who were oriented to the objectives of the project, refresher training and hands-on clinical experience in their districts  -  . The on-the-job experience they had providing mentoring support that was structured for a year and more  , interactions with peers and PHC staff, multiple situations faced could have also contributed to this   .
Thus for a successful nurse mentor training program, key considerations  must be kept in mind. Firstly, interventions must include a balanced mix of both face to face theory and supervised mentored practical hours. Secondly, sufficient time to self-read, experience, discuss, clarify and thus internalise concepts must be provided. Thirdly, for sustained learning over a period of 1 year, additional support through refresher training and supportive supervision visits from senior professionals sensitised to the program could be an added bonus. Nurse mentors thus can be moulded through a short-term targeted training programme to help in improving quality of care of obstetric and newborn care at the public health level  given the right motivation and support.
Careful selection of nurse mentors is crucial keeping in mind their basic competencies, their attitudes and motivation levels. “Mentors are not born but made” is clear from the results of this project. A group of novice nurse professionals if provided with a balanced and focused experiential training, on the field support by sensitised senior specialists, and continued learning opportunities could be developed into professionals who are confident, focused and aware of their role in improving the quality of maternal and newborn care  at the PHCs.
The project was funded by the Bill and Melinda Gates foundation. St John’s Research Institute was a partner with Karnataka Health Promotion Trust (KHPT/FC/NRH/GA/14-15/JUL-01), and the University of Manitoba, Canada, the primary grant recipient. Due acknowledgement goes to the extended team of specialists (doctors and nurses) from departments of Obstetric and neonatology of St John’s Medical College and Hospital for implementation of the program; the statistician and other project staff of St John’s Research Institute who helped with organising data, analysing data; and nurse mentors themselves for consenting to participate in the assessments.
Conflict of Interest
All authors were given a personnel fee for being part of the project through the funds.