CE  Vol.8 No.10 , August 2017
Co-Curricular Learning and Assessment in New Student Orientation at a Community College
ABSTRACT
With increased public pressure to impact student success, assess student learning and increase completion outcomes at community colleges, mandatory orientation can be an important first step on the path to success. Colleges also need to identify opportunities for assessing co-curricular learning in student affairs programs and functions. This study describes the process of implementing a mandatory new student orientation at a community college in Colorado. The program assessment included data collection at the individual student level to document student learning, and more broadly to demonstrate that the program contributed to increases in student retention outcomes and documented student learning from the beginning of the college transition process. The authors found statistically significant differences in student learning about key on boarding topics following orientation and the new student orientation also contributed to a seventeen percent increase in fall to spring retention of new students.

1. Introduction

In the past two decades, increased attention and resources have been targeted to the needs students have, and the challenges students face as they transition to college (Hunter & Murray, 2007) . This has become an even greater focus as the student population has become increasingly diverse, and more first-generation students are enrolling in college (Heinisch, 2016; Soria, Lingren Clark, & Coffin Koch, 2013) . Orientation plays a pivotal role in transitioning students to post-secondary educational environments (Lotkowski, Robbins, & Noeth, 2004) . Transition to college programs need to include understanding processes, communication, tasks, systems, planning, goal setting, and college expectations, as well as learning goals (Heinisch, 2016; Shugart & Romano, 2008) . In addition, orientation programming needs to gradually transfer competence and responsibility to the students (Quaye & Harper, 2014; Shugart & Romano, 2008) . Student success and retention begin with, and depend on this transition being successful, beginning with orientation, but often continuing through a student success or first-year experience program (Hunter & Murray, 2007) . Orientation, supplemental instruction, student success courses, and learning communities are all ways to achieve these outcomes, and increase student engagement on campus (Shugart & Romano, 2008) .

Orientation is the first opportunity for the students and the institution to learn about each other, and to begin developing relationships that promote student development, success, and completion. Not only is orientation the first opportunity to convey institutional expectations, sometimes it is the only opportunity to do so (Mayhew, Vanderlinden, & Kim, 2010) . To remain globally competitive, we must ensure that a higher number of adults earn college credentials (Lotkowski et al., 2004; Soares, 2013) . This means both recruiting and retaining more students in college. It also means that higher education currently has a very large enrollment of students who have varied needs and educational backgrounds (Heinisch, 2016; Hunter & Murray, 2007) . With the goal of serving students who might not otherwise attend college, community colleges have an especially important role in meeting the nation’s workforce and college completion goals (Harbour & Smith, 2016; O’Gara, Mechur, & Hughes, 2009) . Almost half of all the students enrolled in higher education in this country are enrolled at community colleges (Mullin, 2012) .

Orientation and transition to college is an important first step in successfully serving larger numbers of diverse students in higher education. Understanding student needs, acknowledging cultural differences and multiple identities, defining student success, clarifying degree and program requirements, and preparing for effective teaching are all key outcomes for student affairs professionals who develop, lead, and teach college transition programming in all its various forms at community colleges (Hunter & Murray, 2007; Soria et al., 2013) .

2. Review of Literature

2.1. Orientation

Orientation means an effort to promote the successful transition to college for new students. The purpose of orientation is to ease the transition to college, and to convey key institutional expectations (Boening & Miller, 2005; Hollins, 2009; Hunter, 2006; Mayhew et al., 2010) . Orientation provides co-curricular support for student success (Mayhew et al., 2010) . It is the opportunity to set the tone for new students. College transition programming is one of the few opportunities to bring an entire class together (Boening & Miller, 2005; Mayhew et al., 2010) . Effective orientation programs are intentional and comprehensive (Fontaine, 2014; Hunter, 2006; Turner & Thompson, 2014) . Orientation programs reinforce that students are valued by the institution, and that their academic success will be supported (Lotkowski et al., 2004; Turner & Thompson, 2014) .

In order to successfully adjust personally and academically to college, students need to gain an expanded sense of self beyond what they came to college with, including a strong identification with academic domains and academically rigorous programs (Blake, 2007; Hollins, 2009; Hunter & Murray, 2007; Tovar, 2015) . Academic self-confidence, achievement motivation, clear goals, drive to succeed, ability to manage external demands, and self-advocacy are non-aca- demic factors that contribute to college success (Lotkowski et al., 2004; Martin, Galentino, & Townsend, 2014) . Orientation programs can impact these factors that contribute to retention by addressing their preparedness, identification with the college, connecting the academic and social culture of the institution, and helping to set academic goals and expectations (Lotkowski et al., 2004; Martin et al., 2014; Mayhew et al., 2010; Singer, 2016) . Orientation programming establishes an important connection to the college that contributes to retention, demonstrates institutional commitment, and provides students with social support networks (Lotkowski et al., 2004; Martin et al., 2014; Mayhew et al., 2010; Singer, 2016) .

Orientation provides the opportunity to emphasize, and increase student awareness about the differences in college expectations from their previous educational and work experiences (Degen & Sheldahl, 2007; Gibbons & Woodside, 2014; Hollins, 2009; Sandeen & Barr, 2014) . New student orientation highlights opportunities for student involvement on campus, emphasizes the importance of study skills, and provides important information on degree requirements and course scheduling (Degen & Sheldahl, 2007; Gibbons & Woodside, 2014; Sandeen & Barr, 2014) . Orientation is a powerful tool that can help students develop cognitively, academically, and socially (Boening & Miller, 2005; Gibbons & Woodside, 2014; Hollins, 2009; Sandeen & Barr, 2014) .

Over 70% of undergraduates participate in some sort of orientation to the college environment (Boening & Miller, 2005; Eagan, Stolzenberg, Ramirez, Aragon, Suchard, & Hurtado, 2013; Hollins, 2009) . Research has documented orientation outcomes including increased self-esteem, establishment of social support networks, and increased retention (Boening & Miller, 2005; Gray, Vitak, Easton, & Ellison, 2013; Hunter, 2006; Jamelske, 2009; Mayhew et al., 2010) . Student participation in orientation is a significant positive predictor of personal, social, and academic adjustment to college (Boening & Miller, 2005; Gray et al. 2013) .

2.2. Community College Orientation Programs

Hollins (2009) states, “The most underemphasized strategy for promoting student success at community colleges is the development and implementation of an intentional, comprehensive approach to orienting new students to the college environment”. (p. 15). Community colleges often find successfully transitioning students to college to be challenging (Fontaine, 2014; Hollins, 2009) . In addition, community colleges struggle to reach first year retention rates of fifty percent (Derby & Smith, 2004; Hollins, 2009; Permzadian & Credé, 2016) . Community colleges have often failed to make the connection between orientation, and student retention, success, and graduation (Derby & Smith, 2004; Martin et al., 2014; Permzadian & Credé, 2016) . Rather than viewing orientation as the opportunity to set students on the path to success, and impact student completion outcomes, community colleges are too often concerned that requiring participation in orientation presents an enrollment barrier for students. Without orientation, students often lack comprehensive information about curricula, requirements, institutional policies, services, and resources (Hollins, 2009) . There is also a strong link between orientation, and student persistence, retention, and graduation, which are strategic goals for most community colleges (Derby & Smith, 2004; Fontaine, 2014; Martin et al., 2014; Mayhew et al., 2010) .

Because a college degree is more and more necessary to be competitive in the job market, many young people, and adults with little or no college experience are enrolling in colleges in record numbers (Darling-Hammond, 2015; Jamelske, 2009) . This is also the reason that community colleges have the highest number of diverse enrollments, face greater demands and expectations, and consequently have a bigger responsibility for understanding cultural differences, and creating social justice through education (Boening & Miller, 2005; Hunter & Murray, 2007; Kisker, 2005; Rhoads & Valadez, 2016) . In community colleges in particular, orientation can increase and promote diversity and social justice through orientation (Banks, 2015; Ganser, 2012) . This is based on research suggesting interactions with diverse peers, and participation in diversity-related and activities outside of class inspire students to challenge their mental models of race and ethnicity, and promote inclusion and social justice (Zuniga, Williams, & Berger, 2005) . Student orientation leaders in particular, can facilitate connections for students, especially on non-residential campuses. They represent the wide diversity of students at community colleges including first-generation, transfer students, post-traditional students, and families (Ganser, 2012) . For community colleges, with the diverse populations that they serve, many who face significant financial and academic challenges, orientation is essential to their successful transition to, and engagement in college (Ganser, 2012; Hunter & Murray, 2007; Jacoby, 2014) .

The differences for community colleges include the fact that they are more often non-residential campuses with smaller classes, more evening and career-oriented programs, and have fewer opportunities for co-curricular programming (Boening & Miller, 2005; Hughes, 2015; Jacoby, 2014) . Students at community colleges need lower cost educational options, need to be able to work full-time, and often have remediation needs (Boening & Miller, 2005; Majer, 2017) . This results in lower demand for extra-curricular activities, and fewer opportunities for co-curricular programming (Boening & Miller, 2005; Hughes, 2015) . Yet, community college students need, and can benefit from curricular and co-curricular programming that helps them transition to the college environment (Hughes, 2015; Jamelske, 2009) . This makes orientation even more important, and builds the case for mandatory programs, to compel students who are balancing work, family, and school to participate in orientation that sets the tone for their success.

Obstacles to effective orientation at community colleges include the length of time available and adequate structures to support such programming (Boening & Miller, 2005) . However, the power to influence student behavior is still significant. At community colleges orientation can also help promote a more inclusive, diverse environment (Boening & Miller, 2005; Hunter & Murray, 2007) . It can have occupational relevance for students in addition to fulfilling its main function of setting institutional expectations (Boening & Miller, 2005) . Because of the inherent benefits, most community colleges offer some kind of an orientation program. However, the format, content, requirements, and quality vary greatly from school to school.

In addition, there is very little research on the effects of orientation on student outcomes, defined as academic success and retention, at community colleges (Boening & Miller, 2005; Hollins, 2009; Jamelske, 2009; Mayhew et al., 2010) . Student perceptions of the impact of orientation have been examined. In one study, students reported that new student orientation strongly influenced their expectations of college (Mayhew et al., 2010) . Another study found that for transfer students, orientation helped most with academic adjustment, while orientation helped diverse students with social adjustment to college (Mayhew et al., 2010) . Academic and social adjustment to college correlates positively with academic performance, and personal adjustment (Derby & Smith, 2004; Mayhew et al., 2010) . In addition, participation in orientation has been shown to positively impact student grade point average (GPA) (Jamelske, 2009) . There is some research on particular orientation models. The Student Orientation, Assessment and Registration (SOAR) model works well at community colleges, and has been associated with increased GPA, as well as increased retention at community colleges (Hollins, 2009) . The SOAR model provides a comprehensive orientation, assessment, advising, and registration in a single orientation session (Robinson, Burns, & Gaw, 1996) . Regardless of format, research has determined that orientation is the most effective when there is an identified orientation office on campus, and when the orientation programming has access to critical campus academic and student affairs resources (Mayhew et al., 2010) . This highlights the nature of the community college as an interdependent entity, with relationships between academic affairs and student affairs being central to promoting student success (Dale & Drake, 2005; Sandeen & Barr, 2014) .

2.3. Assessment of Co-Curricular Learning in Student Affairs

In today’s higher education landscape, demographic and political changes require a new approach to the way we administer student affairs programming, especially at publically funded institutions (Blake, 2007; Kezar, Chambers, & Burkhardt, 2015; Kisker, 2005; Kuh, Jankowski, Ikenberry, & Kinzie, 2014; Winston, Creamer, & Miller, 2013) . Both external and internal stakeholders are interested in assessment of student learning, including evidence of student learning in student affairs (Henning, Mitchell, & Maki, 2008; Kezar et al., 2015; Kisker, 2005; Kuh et al., 2014; Ribera, Fernandez, & Gray, 2012) . Additionally, retention of college students requires student affairs professionals to focus more on student learning and academic success through active involvement in the learning process (Blake, 2007; Kuh et al., 2014) . However, in the current higher education environment, budget cuts and organizational structures have eroded the role that student affairs programs once played in student learning (Dale & Drake, 2005; Kezar et al., 2015) . In community colleges in particular, there is a lack of understanding about assessment of student learning in student affairs (Kisker, 2005; Kuh et al., 2014) .

In college, learning occurs through both curricular and co-curricular experiences, thus highlighting the importance of assessment in student affairs programming (Dean, 2015; Henning et al., 2008; Kisker, 2005) . Student affairs professionals need to create social and academic integration into the campus by setting high expectations including in co-curricular student life programming (Blake, 2007; Hunter & Murray, 2007; Skipper & Keup, 2017) . Student affairs professionals are facilitators of student development and learning. They need to see themselves as members of a learning community in which their roles are to teach and facilitate learning (Ellertson & Thoennes, 2007) . In addition, student affairs staff needs to include in their strategic planning support for student success, and assessment of student learning (Kuh et al., 2014; Quaye & Harper, 2014; Shugart & Romano, 2008) .

Colleges need to build meaningful programs of mutual support and cooperation between academic affairs and student affairs all within a context of high expectations for student achievement and success (Blake, 2007; Kezar et al., 2015; Sandeen, 2004) . These high expectations emerge from the collaborative aspects of joint programming between different campus constituencies that in the past have operated separately (Blake, 2007) . We need to move from conflict and adversarial relationships to a culture of cooperation, growth, and learning, and his requires active participation from student affairs (Blake, 2007; Gulley & Mullendore, 2014) . There also needs to be assessment of activities and programs in order to determine their potential to impact student learning (Blake, 2007) . Successfully implemented orientation programming indicates a learning approach to management of the college (Mayhew et al., 2010) . This includes a robust assessment of the learning outcomes.

3. History and Overview of the Orientation Program

The New Student Orientation (NSO) program at the college emerged out of a campus wide enrollment management taskforce in 2011. Based on previous research that community college student programs that were not mandatory did not achieve satisfactory results (Fike & Fike, 2008) , we recommended that the orientation be mandatory for all new, first time to college students. Implementing mandatory orientation for all new students was the number one retention recommendation made by the taskforce, and included in the college’s three-year enrollment management plan. However, it should be noted that not everyone campus believed that it was possible, or feasible to successfully implement a mandatory NSO program on campus. The prevailing attitude among the naysayers was that they would watch, and wait, assuming the initiative would fail. Taking into consideration context, the following year, the Vice President of Student Affairs convened an implementation committee to review models, design, and implement a mandatory new student orientation program that was tailored not just to community college students, but that took into consideration the culture, student characteristics, and nuances of our particular campus. The committee was inclusive, and cross institutional to purposely include not only student affairs, academic affairs, and institutional research, but to also address the perspectives of those who were not entirely supportive.

The mandatory orientation was implemented in the spring and summer of 2012 for all new, first time to college students enrolling for fall of 2012. Since that time, the Center for Recruitment and Orientation, which houses the orientation program has collected data to assess and refine the orientation program, and has made changes, and adjustments to the schedule based on student, staff, and faculty evaluations. The goal of the program, however, has remained the same for the past five years. We developed the topics for the sessions based on many previous studies on the goals and purpose of orientation to cover college expectations, student success strategies, support programs, and opportunities for involvement (Boening & Miller, 2005; Hollins, 2009; Hunter, 2006; Mayhew et al., 2010; Shugart & Romano, 2008) . By the end of orientation, students have been introduced to campus expectations, student support services, federal “student right to know” requirements, campus technology, learning management systems, and program requirements. In addition, by the end of the program students have completed the assessment testing, met with an advisor, and registered for their first semester of courses. Throughout the five-year history of the program, regardless of the iterative process that continues to characterize orientation, the founding principles have remained intact. Those are a laser focus on student learning and retention, commitment to cross campus collaboration to implement and deliver the program, clearly outlining campus expectations for students, and engaging students from initial enrollment process to promote student development, success, and completion.

4. Methodology

The assessment of new student orientation was designed based on our experience with previous program assessment, and from two bodies of literature. First, we reviewed the literature on assessment of student learning in student affairs. Second our assessment methodology reflected the literature regarding self-report assessment of training outcomes, with specific emphasis on controlling for a common phenomenon in self-report evaluation called response shift bias. Finally, we considered the college completion agenda. The shift from access to completion is especially noteworthy for community colleges (Blake, 2007) . As is the subsequent importance of student outcomes including first year retention, and academic success to most higher education enrollment programs (Jamelske, 2009; Martin et al., 2014) . Our assessment plan included assessment of student learning in orientation, as well as measuring the first-year retention and GPA of students who completed orientation compared to the overall college statistics for these metrics.

Metzoff (1981a) identifies the two methods of measuring training effectiveness; the participant self-report questionnaire, and the behavioral observation measure. Using the traditional pre/post method for self-report assessments has long been considered the least effective and reliable method of measuring training outcomes because 1) on the pre-assessment participants may have limited knowledge of the construct being measured and therefore do not have the reference to adequately assess their prior knowledge, and 2) they tend to underestimate training benefits (Drennan & Hyde, 2008; Metzoff, 1981a ; Rockwell & Kohn, 1989; Schwartz & Sprangers, 2010 ).

One of the researchers previously worked on a large-scale, statewide training project that initially used pre-tests and post-tests to assess training outcomes including a coding system so we would be able to match the pre and post assessments. An analysis of those data revealed that some participants reported less knowledge and fewer skills after the training―an unlikely circumstance (French, 2002) . This decreased perception of knowledge and skills was obviously a concern. In an effort to understand this shift, participants were questioned about this response. Those interviewed explained that when they completed the post-tests, they recognized that they had over-rated their own knowledge and skills on the pre-test (French, 2002) . Unfortunately, it was too late to go back and change the pre-test at that point.

The tendency for the participant to redefine his/her frame of reference after training, called response-shift bias, has been found in numerous studies using pre and post assessments in a variety of training situations, and causes pretest-posttest comparisons to be inaccurate (Blome & Augistin, 2016; Davis, 2002; Drennan & Hyde, 2008; Howard, 1980; Metzoff, 1981b ; Schwartz & Sprangers, 2010; Umble, Upshaw, Orton, & Matthews, 2000) . Response-shift bias means that the participant has shifted his/her internal definition of the criteria by which to judge the appropriate rating of an item because the training has impacted his/her frame of reference (Blome & Augustin, 2016; Davis, 2002; Drennan & Hyde, 2008; Howard, 1980; Metzoff, 1981b ; Schwartz & Sprangers, 2010; Umble et al., 2000) .

To control this source of contamination in self-report measures, respondents can be asked to reflectively or retroactively assess their skill levels prior to, and subsequent to training. When they complete both the pre and the post segments of the self-assessment at the same time, they use the same criteria, thus providing a more accurate portrayal of their perceptions. Instruments that measure perceptions this way are variously called post-then-pre assessments, reflective post-then assessments, or retrospective pre-tests. They are considered more accurate in these situations than pretests followed by posttests because the participant is using the same frame of reference to judge the construct being measured (Davis, 2002; Drennan & Hyde, 2008; Moser, Kemter, Wachsmann, Köver, & Soucek, 2016; Rockwell & Kohn, 1989) . This approach is also a more culturally sensitive approach to research because it controls for difference in response styles, and response shift levels due to age, educational level, gender, socio-eco- nomic status, ethnicity, position and nature of the program participation (voluntary vs. mandatory) (Awad, Patall, Rackley, & Reilly, 2016; Klatt & Taylor-Powell, 2005) .

Over 65% of the students at the college are first-generation students, and they may have little prior knowledge of the college experience and expectations. Based on this knowledge of our student body, and the researchers’ previous experience with training assessment we used the literature on response shift bias to design a retrospective pre then post-test based on the learning outcomes for new student orientation. Our goal was to demonstrate student learning in orientation.

A retrospective pretest-posttest evaluation instrument was administered at the conclusion of the orientation. As part of the last session of the new student orientation, students were asked to complete an evaluation to indicate their level of knowledge in the key learning outcomes before and after they participated in the orientation, on a scale from 1 to 5 where 1 = Strongly Disagree and 5 = Strongly Agree. As designed, the retrospective pretest evaluation allowed the students to indicate how much their knowledge has changed as a result of the orientation. There were twenty seven learning outcomes we assessed.

The results of the post-then-pre orientation evaluations were analyzed using one-way repeated measures analysis of variance (ANOVA) for the overall effect of new student orientation, and paired sample t-tests for each individual topic covered in the orientation sessions.

5. Results

5.1. First Orientation Cycle―Retrospective Pre-Test Results

Data was collected from 604 students who completed the college’s new student orientation assessment. The assessment was a post-then-pre evaluation which contained 27 items concerning the new students’ knowledge and skills, as well as confidence levels prior to and after completion of the college’s new student orientation. The data that was collected from the student respondents was then analyzed. The statistical method employed for the assessment was a one-way repeated measures analysis of variance (ANOVA). Results of the analysis indicated significant main effects for the orientation program, F(1603) = 2014.37, p = 0.00. The means and standard deviations for pre and post orientation scores are presented in the table below (Table 1).

Since the overall analysis from the ANOVA revealed a significant result, we conducted pairwise comparisons on each of the 27 items to determine which means differed significantly from each other. The statistical technique used to analyze the 27 items was a paired-samples t test. The item-by-item analyses from the post/then pre surveys demonstrated that the mean (M) for the post-orienta- tion data was significantly higher than the mean from the pre-orientation with a significance level of p < 0.01 on each of the 27 survey items examined. The results demonstrate compelling support that the new student orientation significantly increased the participants’ knowledge, skill levels, and confidence perception (Table 2).

Table 1. First Orientation Cycle Mean and Standard Deviation Data.

Table 2. First Orientation Cycle Post/Then Survey Analysis.

* t test significant at p < .01 on a two-tailed test.

5.2. Second Orientation Cycle―Retrospective Pre-Test Results

Data from post-then-pre surveys collected from 185 student survey respondents were analyzed in order to determine the knowledge, skill and confidence level prior to and after completing the college’s new student orientation. A one-way repeated measures analysis of variance indicated significant main effects for the orientation program, F(1184) = 864.15, p = .00. The means and standard deviations for pre and post orientation scores are presented in the table below (Table 3).

Since the overall analysis from the ANOVA revealed a significant result, we conducted pairwise comparisons on each of the 27 items to determine which means differed significantly from each other. The statistical technique used to analyze the 27 items was a paired-samples t test. The item-by-item analyses from the retrospective pre-tests demonstrated that the mean (M) for the post-orienta- tion data was significantly higher than the mean from the pre-orientation with a significance level of p < 0.01 on each of the 27 survey items examined. The results demonstrate compelling support that the new student orientation significantly increased the participants’ knowledge, skill levels, and confidence perception (Table 4).

5.3. Retention and Persistence Results

For students who participated in the revised and expanded orientation there was a fall to spring retention rate of 83%. The average overall college retention rate from fall to spring during that same time period was 66%. Students who participated in new student orientation were retained at a rate 17% higher than the average student at the college. In addition, for students who participated in new student orientation their average GPA was a 2.95 compared to the college average student GPA of 2.5. Students who participated in new student orientation earned GPAs that were 0.45 higher than the average student at the college.

6. Conclusions

Viewing the implementation of this mandatory orientation program through the lens of its significant role in transitioning students to college, our new student orientation program documented through attention to context, effective planning, and assessment that the program began to familiarize students with academic requirements and expectations, assisted students in learning about the different services available to them on campus, aided students in setting personal goals, and educated them in the most effective ways to achieve these goals in four key ways.

Table 3. Second Orientation Cycle Mean and Standard Deviation Data.

Table 4. Second Orientation Cycle Post/Then Survey Analysis.

*t test significant at p < .01 on a two-tailed test.

First, in examining the role that transition programming has in social integration to college, the results of our assessment demonstrate this orientation program achieved the goal of introducing the student to the college and beginning to develop relationships that promoted social integration into the campus. This program fulfilled a key role outlined in the literature as providing co-curricular support for student success (Gale & Parker, 2014; Mayhew et al., 2010) .

Second, using student orientation leaders, this orientation program assisted in promoting social integration by helping form personal relationships with other students. Because the orientation leaders represented the diversity of our student body, orientation also introduced students to the diversity of the campus, and assisted in promoting diversity and expanding multicultural competence. Orientation allowed the students to get a feel for the campus culture, and helped engage them in the campus life, including highlighting the variety of campus organizations and how to get involved. Finally, orientation assisted students in learning how to network with others on campus.

Third, we began with an overriding belief that effective orientation programs are intentional and comprehensive. Therefore, the college’s New Student Orientation program was designed intentionally, and collaboratively. As a result, it not only helped students begin to become familiar with the different terminology and language used at the college, and begin to understand their responsibilities as a member of the college community, but also helped to convince the skeptics that this was an important first step in the student success and completion initiatives being implemented on the campus. In addition, we can document that the orientation assisted in promoting a positive attitude and an excitement for learning. In a more practical sense, the orientation assisted students in developing writing skills, study skills, time management skills and acclimating them to the learning management system. This was achieved in part by including faculty in the orientation program, thus helping students begin to develop a stronger identification with academic programs. This approach aligned with best practice and our focus on student success (Braskamp, Trauvetter, & Ward, 2016) .

Fourth and probably most important are the student learning and retention outcomes that we were able to document by designing a thoughtful, inclusive implementation and assessment process. By including both assessment of student learning, and long-term data analysis, our implementation process supported previous research that indicates one of the roles of orientation is increasing awareness of the self as a learner, establishing social support networks, and increasing retention (Boening & Miller, 2005; Hunter, 2006; Jamelske, 2009; Mayhew et al., 2010; Permzadian & Credé, 2016) . The significant assessment process built into the program also helped create a convincing case for dedicating increased resources to the program. Our assessment results tied the orientation outcomes to major college and division strategic goals. This resulted in being able to acquire more resources to support the program. This approach helps to demonstrate to students the centrality to the college’s mission (Braskamp et al., 2016) .

7. Implications for Future Research

This study has demonstrated that data can be collected at orientation that can build a foundation for colleges to track students through their academic career to continue to build a case for the correlation between orientation, retention, and completion. This provides important structure for future research on the importance of mandatory orientation for helping community college achieve college, state and federal goals for student success and completion. Additional elements could be added to the assessment of the effectiveness of new student orientation programming, and students could also be tracked to examine the impact of orientation on grade point average throughout a college career. Finally, transfer is one of the important community college missions (Hirt, 2006; Jenkins & Fink, 2016) . Future research could involve the collaboration with key transfer institutions to track the success of transfer students, by utilizing data sharing agreements, and data from the National Student Clearinghouse.

8. Implications for Future Practice

The results of this study present essential guidance on how to construct new student orientations not only structurally but also instructionally. Strategies used in this orientation program help illuminate a significant first step in helping community colleges respond to the ever-increasing diversity of the student body, as well as increasing external pressure to improve student success and completion outcomes. Findings from this study may also be utilized to involve other groups on campus. While orientation is usually the province of student affairs (Hunter & Linder, 2005) , it has emerged as a favorably horizontal structure in higher education that fosters valuable collaboration between student and academic affairs (Greenfield, Keup, & Gardner, 2013; Keeling, Underhile, & Wall, 2007) . This study illuminates the significance of taking a learning outcomes approach to assessing programming. It also promotes orientation programming as a primary instrument that assists students’ transition to college.

Cite this paper
Sandoval-Lucero, E. , Antony, K. and Hepworth, W. (2017) Co-Curricular Learning and Assessment in New Student Orientation at a Community College. Creative Education, 8, 1638-1655. doi: 10.4236/ce.2017.810111.
References
[1]   Awad, G. H., Patall, E. A., Rackley, K. R., & Reilly, E. D. (2016). Recommendations for Culturally Sensitive Research Methods. Journal of Educational and Psychological Consultation, 26, 283-303.
https://doi.org/10.1080/10474412.2015.1046600

[2]   Banks, J. A. (2015). Cultural Diversity and Education. New York, NY: Routledge.

[3]   Blake, J. H. (2007). The Crucial Role of Student Affairs Professionals in the Learning Process. New Directions for Student Services, 2007, 65-72.
https://doi.org/10.1002/ss.234

[4]   Blome, C., & Augustin, M. (2016). Measuring Change in Subjective Wellbeing: Methods to Quantify Recall Bias and Recalibration Response Shift (No. 2016/12). HCHE Research Paper.

[5]   Boening, C. H., & Miller, M. T. (2005). New Student Orientation Programs Promoting Diversity. The Community College Enterprise, 11, 41-50.

[6]   Braskamp, L. A., Trautvetter, L. C., & Ward, K. (2016). Putting Students First: How Colleges Develop Students Purposefully. New York, NY: John Wiley & Sons.

[7]   Dale, P. A., & Drake, T. M. (2005). Connecting Academic and Student Affairs to Enhance Student Learning and Success. New Directions for Community Colleges, 2005, 51-64.
https://doi.org/10.1002/cc.205

[8]   Darling-Hammond, L. (2015). The Flat World and Education: How America’s Commitment to Equity Will Determine Our Future. New York, NY: Teachers College Press.

[9]   Davis, G. A. (2002). Using a Retrospective Pre-Post Questionnaire to Determine Program Impact. In The Annual Meeting of the Mid-Western Educational Research Association.

[10]   Dean, K. L. (2015). Understanding Student Success by Measuring Co-Curricular Learning. New Directions for Institutional Research, 2014, 27-38.
https://doi.org/10.1002/ir.20113

[11]   Degen, G., & Sheldahl, E. (2007). The Many Hats of Teaching in Small Colleges: The Seamless Web of Student and Academic Affairs. New Directions for Student Services, 2007, 47-56.
https://doi.org/10.1002/ss.232

[12]   Derby, D. C., & Smith, T. (2004). An Orientation Course and Community College Retention. Community College Journal of Research and Practice, 28, 763-773.
https://doi.org/10.1080/10668920390254771

[13]   Drennan, J., & Hyde, A. (2008). Controlling Response Shift Bias: The Use of the Retrospective Pre-Test Design in the Evaluation of a Master’s Programme. Assessment & Evaluation in Higher Education, 33, 699-709.
https://doi.org/10.1080/02602930701773026

[14]   Eagan, K., Stolzenberg, E. B., Ramirez, J. J., Aragon, M. C., Suchard, M. R., & Hurtado, S. (2013). The American Freshman: National Norms Fall 2013. Los Angeles, CA: Higher Education Research Institute.

[15]   Ellertson, S., & Thoennes, K. V. (2007). Reframing Teaching and Learning: Lessons from Learning Communities for Student Affairs. New Directions for Student Services, 2007, 35-46.
https://doi.org/10.1002/ss.231

[16]   Fike, D. S., & Fike, R. (2008). Predictors of First-Year Student Retention in the Community College. Community College Review, 36, 68-88.
https://doi.org/10.1177/0091552108320222

[17]   Fontaine, K. (2014). Effects of a Retention Intervention Program for Associate Degree Nursing Students. Nursing Education Perspectives, 35, 94-99.
https://doi.org/10.5480/12-815.1

[18]   French, N. K. (2002). Paraeducator Supervision Academy (PSA) Outreach Project. Final Performance Report.

[19]   Gale, T., & Parker, S. (2014). Navigating Change: A Typology of Student Transition in Higher Education. Studies in Higher Education, 39, 734-753.
https://doi.org/10.1080/03075079.2012.721351

[20]   Ganser, S. L. (2012). Where It All Began: Peer Education and Leadership in Student Services. New Directions for Higher Education, 2012, 17-29.
https://doi.org/10.1002/he.20003

[21]   Gibbons, M. M., & Woodside, M. (2014). Addressing the Needs of First-Generation College Students: Lessons Learned from Adults from Low-Education Families. Journal of College Counseling, 17, 21-36.
https://doi.org/10.1002/j.2161-1882.2014.00045.x

[22]   Gray, R., Vitak, J., Easton, E. W., & Ellison, N. B. (2013). Examining Social Adjustment to College in the Age of Social Media: Factors Influencing Successful Transitions and Persistence. Computers & Education, 67, 193-207.

[23]   Greenfield, G. M., Keup, J. R., & Gardner, J. N. (2013). Developing and Sustaining Successful First-Year Programs: A Guide for Practitioners. New York, NY: John Wiley & Sons.

[24]   Gulley, N. Y., & Mullendore, R. H. (2014). Student Affairs and Academic Affairs Collaborations in the Community College Setting. Community College Journal of Research and Practice, 38, 661-673.
https://doi.org/10.1080/10668926.2011.585115

[25]   Harbour, C. P., & Smith, D. A. (2016). The Completion Agenda, Community Colleges, and Civic Capacity. Community College Journal of Research and Practice, 40, 100-112.
https://doi.org/10.1080/10668926.2014.996923

[26]   Heinisch, B. P. (2016). Small Fish Out of Water: Rural First-Generation Student Experience at a Large University. The Journal of College Orientation and Transition, 24, 21-33.

[27]   Henning, G. W., Mitchell, A. A., & Maki, P. L. (2008). The Assessment Skills and Knowledge Standards: Professionalizing the Work of Assessing Student Learning and Development. About Campus, 13, 11-17.
https://doi.org/10.1002/abc.259

[28]   Hirt, J. (2006). Where You Work Matters: Student Affairs Administration at Different Types of Institutions. Lanham, MA: University Press of America.

[29]   Hollins Jr, T. N. (2009). Examining the Impact of a Comprehensive Approach to Student Orientation. Inquiry, 14, 15-27.

[30]   Howard, G. S. (1980). Response-Shift Bias: A Problem in Evaluating Interventions with Pre/Post Self-Reports. Evaluation Review, 1, 93-106.
https://doi.org/10.1177/0193841X8000400105

[31]   Hughes, A. R. (2015). Leadership Development among Traditional-Age Community College Students. Northeastern University.

[32]   Hunter, M. S. (2006). Fostering Student Learning and Success through First-Year Programs. Peer Review, 8, 4-7.

[33]   Hunter, M. A., & Linder, C. W. (2005). First-Year Seminars. In M. L. Upcraft, J. N. Gardner, B. O. Barefoot, & Associates (Eds.), Challenging and Supporting the First-Year Student: A Handbook for Improving the First Year of College (pp. 275-291). San Francisco, CA: Jossey-Bass.

[34]   Hunter, M. A., & Murray, K. A. (2007). New Frontiers for Student Affairs Professionals: Teaching and the First-Year Experience. New Directions for Student Services, 2007, 25-34.
https://doi.org/10.1002/ss.230

[35]   Jacoby, B. (2014). Engaging Commuter and Part-Time Students. In S. R. Harper, & S. J. Quaye (Eds.), Student Engagement in Higher Education: Theoretical Perspectives and Practical Approaches for Diverse Populations (pp. 289-305). New York, NY: Taylor & Francis.

[36]   Jamelske, E. (2009). Measuring the Impact of a University First-Year Experience Program on Student GPA and Retention. Higher Education, 57, 373-391.
https://doi.org/10.1007/s10734-008-9161-1

[37]   Jenkins, D., & Fink, J. (2016). Tracking Transfer: New Measures of Institutional and State Effectiveness in Helping Community College Students Attain Bachelor’s Degrees. Community College Research Center, Teachers College, Columbia University.

[38]   Keeling, R. P., Underhile, R., & Wall, A. F. (2007). Horizontal and Vertical Structures: The Dynamics of Organization in Higher Education. Liberal Education, 93, 22-31.

[39]   Kezar, A., Chambers, A. C., & Burkhardt, J. C. (2015). Higher Education for the Public Good: Emerging Voices from a National Movement. New York, NY: John Wiley & Sons.

[40]   Kisker, C. B. (2005). Key Resources for Student Affairs Professionals in Learning-Centered Community Colleges. New Directions for Community Colleges, 2005, 89-97.
https://doi.org/10.1002/cc.208

[41]   Klatt, J., & Taylor-Powell, E. (2005). Using the Retrospective Post-then-Pre Design, Quick Tips 27. Program Development and Evaluation. Madison, WI: University of Wisconsin-Extension.

[42]   Kuh, G. D., Jankowski, N., Ikenberry, S. O., & Kinzie, J. L. (2014). Knowing What Students Know and Can Do: The Current State of Student Learning Outcomes Assessment in US Colleges and Universities. Urbana, IL: National Institute for Learning Outcomes Assessment.

[43]   Lotkowski, V. A., Robbins, S. B., & Noeth, R. J. (2004). The Role of Academic and Non-Academic Factors in Improving College Retention. Iowa City, IA: ACT Policy Report.

[44]   Majer, J. M. (2017). Community Colleges. In J. J. Viola, & O. Glantsman (Eds.), Diverse Careers in Community Psychology (p. 189). Oxford: Oxford University Press.
https://doi.org/10.1093/acprof:oso/9780190457938.003.0016

[45]   Martin, K., Galentino, R., & Townsend, L. (2014). Community College Student Success: The Role of Motivation and Self-Empowerment. Community College Review, 42, 221-241.
https://doi.org/10.1177/0091552114528972

[46]   Mayhew, M., Vanderlinden, K., & Kim, E. (2010). A Multi-Level Assessment of the Impact of Orientation Programs on Student Learning. Research in Higher Education, 51, 320-345.
https://doi.org/10.1007/s11162-009-9159-2

[47]   Metzoff, B. (1981a). How to Get Accurate Self-Reports of Training Outcomes. Training and Development Journal, 35, 56-61.

[48]   Metzoff, B. (1981b). Pre-then-Post Testing: A Tool to Improve the Accuracy of Management Training Program Evaluation. NSPI Journal, 20, 10-16.

[49]   Moser, K., Kemter, V., Wachsmann, K., Kover, N. Z., & Soucek, R. (2016). Evaluating Rater Training with Double-Pretest One-Posttest Designs: An Analysis of Testing Effects and the Moderating Role of Rater Self-Efficacy. The International Journal of Human Resource Management, 27, 1-23.
https://doi.org/10.1080/09585192.2016.1254102

[50]   Mullin, C. M. (2012). Why Access Matters: The Community College Student Body (Policy Brief 2012-01PBL). Washington DC: American Association of Community Colleges.

[51]   O’Gara, L., Mechur, K., & Hughes, K. (2009). Student Success Courses in the Community College: An Exploratory Study of Student Perspectives. Community College Review, 36, 195-218.
https://doi.org/10.1177/0091552108327186

[52]   Permzadian, V., & Credé, M. (2016). Do First-Year Seminars Improve College Grades and Retention? A Quantitative Review of Their Overall Effectiveness and an Examination of Moderators of Effectiveness. Review of Educational Research, 86, 277-316.
https://doi.org/10.3102/0034654315584955

[53]   Quaye, S. J., & Harper, S. R. (2014). Student Engagement in Higher Education: Theoretical Perspectives and Practical Approaches for Diverse Populations. New York, NY: Routledge.

[54]   Rhoads, R. A., & Valadez, J. R. (2016). Democracy, Multiculturalism, and the Community College: A Critical Perspective (Vol. 1081). New York, NY: Routledge.

[55]   Robinson, D. A., Burns, C. F., & Gaw, K. F. (1996). Orientation Programs: A Foundation for Student Learning and Success. New Directions for Student Services, 1996, 55-68.
https://doi.org/10.1002/ss.37119967507

[56]   Rockwell, S. K., & Kohn, H. (1989). Post-then-Pre Evaluation. Journal of Extension, 27, 19-21.

[57]   Ribera, T., Fernandez, S., & Gray, M. (2012). Assessment Matters: Considering the Scholarship of Teaching and Learning in Student Affairs. About Campus, 16, 25-28.
https://doi.org/10.1002/abc.20084

[58]   Sandeen, A. (2004). Educating the Whole Student: The Growing Academic Importance of Student Affairs. Change: The Magazine of Higher Learning, 36, 28-33.
https://doi.org/10.1080/00091380409605577

[59]   Sandeen, A., & Barr, M. J. (2014). Critical Issues for Student Affairs: Challenges and Opportunities. New York, NY: John Wiley & Sons.

[60]   Schwartz, C. E., & Sprangers, M. G. (2010). Guidelines for Improving the Stringency of Response Shift Research Using the then Test. Quality of Life Research: An International Journal of Quality of Life Aspects of Treatment, Care and Rehabilitation, 19, 455-464.
https://doi.org/10.1007/s11136-010-9585-9

[61]   Shugart, S. C., & Romano, J. C. (2008). Focus on the Front Door of the College. New Directions for Community Colleges, 2008, 29-39.
https://doi.org/10.1002/cc.343

[62]   Singer, J. (2016). The Effect of a First Year Experience Program on Student Retention in Community College (p. 25). Counselor Education Capstone.
http://digitalcommons.brockport.edu/edc_capstone/25

[63]   Skipper, T. L., & Keup, J. R. (2017). The Perceived Impact of Peer Leadership Experiences on College Academic Performance. Journal of Student Affairs Research and Practice, 54, 95-108.
https://doi.org/10.1080/19496591.2016.1204309

[64]   Soares, L. (2013). Post-Traditional Learners and the Transformation of Postsecondary Education: A Manifesto for College Leaders (p. 118). Washington DC: American Council on Education.

[65]   Soria, K., Lingren Clark, B., & Coffin Koch, L. (2013). Investigating the Academic and Social Benefits of Extended New Student Orientations for First-Year Students. The Journal of College Orientation and Transition, 20, 33-45.

[66]   Tovar, E. (2015). The Role of Faculty, Counselors, and Support Programs on Latino/a Community College Students’ Success and Intent to Persist. Community College Review, 43, 46-71.
https://doi.org/10.1177/0091552114553788

[67]   Turner, P., & Thompson, E. (2014). College Retention Initiatives Meeting the Needs of Millennial Freshman Students. College Student Journal, 48, 94-104.

[68]   Umble, K., Upshaw, V., Orton, S., & Matthews, K. (2000). Using the Post-Then Method to Assess Learner Change. Presentation at the AAHE Assessment Conference, Charlotte, 15 June 2000.

[69]   Winston, R. B., Creamer, D. G., & Miller, T. K. (2013). The Professional Student Affairs Administrator: Educator, Leader, and Manager. New York, NY: Routledge.

[70]   Zuniga, X., Williams, E. A., & Berger, J. B. (2005). Action-Oriented Democratic Outcomes: The Impact of Student Involvement with Campus Diversity. Journal of College Student Development, 46, 660-678.
https://doi.org/10.1353/csd.2005.0069

 
 
Top