The issue of teen pregnancy has been addressed through numerous approaches, the most common being the implementation of evidence-based programs (EBPs). As many funding agencies require grantees implement EBPs to have the highest potential impact on their target population, there have been few opportunities for program developers to create new and innovative programs to prevent teen pregnancy. The lack of opportunities to create innovative programs has led to a limited understanding of what makes a program innovative and how to create innovative programs.
The aim of this project was to identify mechanisms that support and enable the development of innovative teen pregnancy prevention programs and to identify characteristics of innovative programs. Before discussing the need for innovative program development in teen pregnancy prevention, we first examine existing EBPs and their strengths and weaknesses. Then, we describe methods used by the Innovative Teen Pregnancy Prevention Programs (iTP3) project to support the development of innovative programs. We also note selected accomplishments of the overall iTP3 project and our grantees. This article contributes to the growing literature on innovative program development at a time when most research and funding is geared toward existing EBPs.
2. Evidence-Based Programs
Evidence-based programs (EBPs) are common health promotion approaches to create positive behavioral change and/or outcomes among intended target populations (Powers et al., 2015) . Due to positive effects demonstrated by research, support and appreciation for EBPs has increased in public health, especially to address the challenge of teen pregnancy prevention in the United States. Given public health concerns and negative impacts on youth achievement resulting from teen pregnancy (Hoffman, 2008; Perper et al., 2010; Penman-Aguilar et al., 2013) , evidence-based interventions have gained ground as common strategies to decrease teen and unplanned pregnancy.
The interest in EBPs also arises from a growing inventory of evidence-based programs and approaches in teen pregnancy prevention. The first EBP list, created in 2007, provided a repository of effective sexual health programs, with evidence of reducing teen pregnancy and associated sexual risk behaviors (Douglas, 2007) . Since then, federal funding initiatives through which the Department of Health and Human Services―Office of Adolescent Health (OAH), Family and Youth Services Bureau (FYSB), and Centers for Disease Control and Prevention (CDC) supported and tested programs, has led to a more current list of evidence-based interventions. This EBP list was developed through rigorous evaluation of programs using measures such as impact on teen pregnancy, teen birth, and sexually transmitted infections (STIs) (OAH, 2016a; OAH, 2016b) . Such programs, supported by government agencies including OAH, Administration for Children and Families (ACF), and the CDC, demonstrate effectiveness in reducing teen pregnancies, STIs, and associated sexual risk behaviors (OAH, 2016a; Maness & Buhi, 2016) .
Among EBPs, most programs were designed to be implemented in middle and high schools, clinics, after school or community-based organizations, and other specialized settings. Approaches varied to include a range from abstinence-based education to comprehensive sexual health education. Program focus (such as youth development), setting, and target population also varied (OAH, 2016a; OAH, 2016b; Kappeler & Farb, 2014) . Many programs include population-based approaches designed for implementation with an entire target population based on race, ethnicity, or other identifiable social characteristics, such as African Americans and Hispanic and/or Latinos (OAH, 2015) . Most programs on the teen pregnancy prevention EBP list impact change at the individual-level by focusing on participants’ knowledge, skills, attitudes, values, and individual access to resources (Goesling et al., 2014) . However, it is important to recognize and address factors that contribute to adolescent pregnancy including: interpersonal dynamics in relationships with partners; social norms, including the onset of sexual behaviors; and systematic factors, such as availability and access to contraceptives (Douglas Jr. & Fenton, 2013) .
From 2010-2014, OAH funded organizations to replicate and evaluate teen pregnancy prevention programs proven to be effective or considered “promising” (Kappeler & Farb, 2014) . The early support of promising programs was an initial investment in advancing teen pregnancy prevention programs. While the field of teen pregnancy prevention has advanced scientific understanding of implementing and replicating EBPs (Koh, 2014; Margolis & Roper, 2014) , gaps still exist in fulfilling target population needs, delivering programs in various settings, using different levels of intervention, and offering non-traditional programs. New delivery methods and targeted programs are necessary to reach teens at higher risk for experiencing teen pregnancy but not reached through traditional programming.
2.1. Need for Innovative Program Development
Innovation has a place in adolescent health program planning beyond its current application of program adoption and dissemination (Wilson et al., 2017) . Fostering and supporting innovation is necessary to keep up with the advances in human behavior and technology, thus the question, “how can the field advance programs that are innovative and engage the target population and impact adolescent pregnancy or birth rates?” remains.
Given the ongoing gaps and disparities in existing EBPs, OAH funded two projects in 2015 to foster and support innovative approaches to teen pregnancy prevention (Kappeler & Farb, 2014) . One project, Innovation Next, focused on technology-based innovations in teen pregnancy prevention, while the other project, the Innovative Teen Pregnancy Prevention Programs (iTP3) project, focused on, “how can we foster innovation in teen pregnancy prevention programs that are innovative in design and approach to impact populations at greatest risk?”.
2.2. The iTP3 Project
Administered at Texas A&M University, the iTP3 project used small grants to support and enable innovative teen pregnancy prevention program development with organizations across the country in various settings. In addition to the small grants, the iTP3 project provided mechanisms to the grantees to enable innovative program development through capacity building and technical assistance, sharing and dissemination, and program planning processes.
Innovative program development is an iterative process to ensure the resulting program is both acceptable to the user and effective. The challenge is establishing a creative and engaging teen pregnancy prevention program for the users that is both different from what already exists and innovative for the field; then testing and disseminating it for adoption before the program loses its innovative characteristics and becomes outdated. Additionally, we know that innovation requires taking risks, and some innovative ideas will fail.
With this in mind, the iTP3 project granted small funding awards (up to $100,000) for 12 months to allow for program development and iteration but avoid long-term investment in programs needing to be re-ideated. At the end of one year, each project had the opportunity to apply for a second year of funding based on how their program progressed over the first funding year. The re-application ensured that only programs deemed to have the highest potential for resulting in effective and innovative programs received continued funding.
The iTP3 project offers one step toward understanding how innovative design strategies and multi-level approaches can advance the field, and how to identify innovative initiatives. Perhaps a new way of thinking can provide opportunities and insights to reach facilitators and participants in a new way.
Small grant programs, such as that discussed here, can serve as models for others interested in working with program developers to ideate and create innovative ideas, that if successful may be tested in a controlled environment. This also represents one approach for establishing a system that actively supports early stages of program development and the recognition that not all program ideas should be invested in from a long-term, high dollar perspective.
The Office of Adolescent Health (OAH) funded the Innovative Teen Pregnancy Prevention Programs (iTP3) project through a grant from 2015-2018. The iTP3 team oversaw the formal selection of small grant recipients to develop innovations in teen pregnancy prevention. The process began with disseminating a formal request for proposals (RFP) to a national audience. Interested applicants submitted complete proposal materials through an electronic portal. Applicants had the opportunity to apply through a traditional grant narrative format or a web-based format to ensure that applicants not well versed in traditional grant applications could still apply. This was due to the thought that smaller, more nimble organizations may have more leverage to break from traditional programmatic norms.
All applications were screened for eligibility once received, then reviewed a panel of professionals serving on the iTP3 Innovation Advisory Committee, with knowledge in various aspects of teen pregnancy prevention and adolescent health. Reviewers identified the top applications using a review protocol. Utilizing a tiered approach, recognizing the need for innovation in multiple areas and at various ecological levels, applicants were selected for funding in Cohort 1 of the iTP3 project.
Ultimately, the successful applicants addressed the needs of an underserved population or planned to develop a program for a unique setting. Additional innovative components and approaches to teen pregnancy prevention were considered. The first cohort of grantees included 15 organizations across the United States (see Figure 1) funded out of 59 applicants. The majority of the small grants were awarded to community-based or health care organizations. All grantees received an award of $100,000 over 12 months to support and enable innovative teen pregnancy prevention programs for the duration of the small grant.
3.1. Capacity Building Assistance
Project teams were encouraged to view their work plan as an iterative document and update it as needed throughout the year. This allowed the teams freedom to listen to their target population and needs assessment results to create a targeted and innovative program, rather than being held to their initial program idea.
Throughout the funding period, grantees received capacity building assistance both on a broad level, and specific to their project. A needs assessment, administered at the beginning of the funding year, provided iTP3 with insight on organizational needs for innovative program development, to guide capacity building activities. Formats included webinars and conference calls with content experts, and information/resource sharing. Each organization participated in monthly calls with a project liaison from the iTP3 team to discuss upcoming
Figure 1. Map indicating state location of iTP3 cohort 1 funded organizations.
activities and identify topics/needs for future activities.
The iTP3 team used a collaborative evaluation framework to encourage partnerships between the iTP3 project team, small grant recipients, and funders to design and implement the program evaluation (Fluhr et al., 2004) . The participatory approach ensured the evaluation would capture the most important data―with the least burden on project teams―while still providing meaningful information about the implementation and outcomes of the iTP3 program.
In order to understand and describe the iTP3 project, process and outcome measures were used in the evaluation. It is important to note that iTP3 aimed to enable and support early teen pregnancy prevention innovations and emphasized implementation and measurable outcomes, not adolescent focused research or outcomes. Thus, the majority of the evaluation focused on implementation and quality improvement measures, which allowed the iTP3 team to monitor progress and document the project’s effectiveness in meeting its stated goals.
Throughout the funding year, grantees provided the iTP3 team monthly project updates and provided progress or changes to their initial work plans. Using this information, iTP3 tracked the progression in the program development process. The monthly updates also provided iTP3 with insight regarding capacity building assistance needs for grantees at both an individual and group level.
Given that most of the small grant programs were in different stages of development, they were responsible for evaluating their own progress and outcomes. Evaluation plans varied and covered a range of domains, as did the programs. Evaluation questions included characteristics about implementation and adaptation, barriers, facilitators, implementation, networks, outcomes, partnerships, reach, sustainability, and performance measures as included in their program plans. For example, some grantees were in the needs assessment stage and conducted interviews and focus groups as part of their evaluation, whereas other programs had a solid plan and utilized multiple data collection strategies.
To understand the scope and range of the iTP3-funded projects and gain insight into potential for innovation and characteristics of innovative settings, the programs are categorized below based on their stage of program development, setting, and target population.
4.1. Innovative Program Ideas and Stages of Development
iTP3 awarded organizations with funding to support program ideas indifferent stages of development (Table 1). The program developers indicated what category of development they believed best suited to their project. The stage of development was confirmed through review by the Innovation Advisory Committee and discussions between the iTP3 team and program developers. Upon
Table 1. Program idea and stage of development.
denotes stage at onset of working with iTP3. denotes state at end of one-year funding cycle.
funding, projects were categorized into four stages of development. Categories of program development including: 1) an idea that had not been developed or transitioning an idea into a promising program; 2) preparing programs for implementation; 3) feasibility testing and/or preparing to pilot program; and 4) final stages of program development. None of the Cohort 1 projects fit in the final stages of program development (category 4). Further, projects were restricted from entering rigorous evaluation as this funding supported development of new programs.
Generally, projects falling into category one engaged their target population(s) to better understand and explore needs in preventing teen pregnancy, and some began with a program idea to engage their target population for further refinement and development. Category two projects began formative program assessments with their target populations and stakeholders to adapt program materials and resources, selected appropriate fidelity measures, created a data collection plan, and conducted preliminary cost analyses. Projects ready to develop a feasibility testing protocol and collect pilot data of their program related to cost, exposure, implementation, and changes or effects on their target population by implementing small scale projects fell into category three.
4.2. Description of Funded Organizations and Program Setting
iTP3 funded various types of organizations to develop innovative teen pregnancy prevention programs as shown in Table 2. Cohort one organizations received awards to develop teen pregnancy prevention programs and were not specifically intended to be the site for program implementation. Table 2 shows the type of organizations funded as well as the proposed program setting. While most programs were intended to be implemented at a physical location, some included a mobile technology piece for participants to access separately. Some programs did not select a location/setting during the application and onboarding stage, so the program could be further developed based on the target population, and
Table 2. Funded organization type and program setting proposed.
HC―Healthcare; RE―Research and Evaluation Institute; HD―Health Department; NP―Non-Profit; S/U―School/University; HC―Healthcare; FCA―Foster Care Agency; SC―School Campus; CBO―Community Based Organization; CC―College Campus; M―Mobile.
accessible and supportive sites. The geographic location of awardees spanned the United States, as shown in Figure 1.
4.3. Characteristics of the Target Populations Served by the Program
By addressing various levels of intervention, a wide range of target populations were engaged in (or the focus of) program development. Most programs intended to target populations that went beyond the adolescents, to include practitioners, parents/caregivers, support staff, community members, trusted mentors, and/or adults. The variety of target populations represented is shown in Table 3 and Table 4. Table 3 describes the target population directly served by the innovative program, while Table 4 describes the target population to be impacted by the program. The target population(s) to be impacted are not exhaustive, it is merely the primary demographic group the programs’ goals are directed towards. It is expected additional demographic groups also be positively impacted as iTP3 required all programs be medically accurate, trauma-informed, culturally and linguistically appropriate, and inclusive of LGBT+ youth. All organizations worked with advisory committees, representative of their stakeholders and target population(s) to provide insight and feedback on programmatic needs and wants. The advisory groups met regularly with program developers to discuss program goals, materials, activities, implementation logistics, anticipated challenges, marketing and recruitment, etc.
Table 3. Characteristics of target population to be directly served by innovative program.
4.4. Level of Intervention
The request for proposals for iTP3 grantees prioritized applications proposing new and innovative approaches that exceeded individual level change. As current teen pregnancy prevention efforts primarily target individual-level change (Farb & Margolis, 2016) , iTP3 emphasized the importance of innovative programs targeting multiple levels, including individual, interpersonal, organizational, community, policy, and system-level change. The levels of intervention targeted by each program are shown in Table 5.
4.5. Selected Accomplishments
The purpose of the iTP3 project was to support and enable innovative program development in various stages of development for underserved populations. Each of the characteristics highlighted above shows a piece of the story for what makes each program innovative. Through the iTP3 mini grants, several accomplishments were evident at the end of cohort one. Following are a few selected accomplishments that illustrate the effectiveness of the iTP3 mini grant model in fostering innovation.
One of the most broad-reaching accomplishments is the incorporation of youth in the program development process. One of the goals of iTP3 was to ensure the programs met the target populations’ needs. According to program design experts and innovative fields such as human centered design, gaining insight from the youth for these teen pregnancy prevention projects was critical
Table 4. Characteristics of target population to be impacted by innovative program.
Table 5. Level of intervention proposed for the innovative program.
(Luma Institute, 2015; Vechakul et al., 2015; Li et al., 2018) . One example of gaining deep insight from the youth included a video-voice project in which participants showed their perspective of the issue at hand through making videos. Other programs utilized innovative strategies to gain insights from the youth they worked with through nontraditional interviews and focus groups sessions.
Another accomplishment included the ability to create and establish new partnerships, as well as ties to systems approaches. One program working with individuals with disabilities expanded their network within their geographic area to gain insight from individuals with disabilities themselves, parents/caregivers of individuals with disabilities, and other adults supporting this target population. Another program increased partnerships that allowed youth greater access to healthcare, while another focused on a systems level approach with foster care staff.
Other programs intended to institutionalize reproductive health care. For example, one hospital system planned to ensure that sexual and reproductive health becomes an important part of the standard care of practice within clinics. This is an example of advancing systems thinking and recognizing that policies, systems, and environments influence important contextual factors and go beyond individual level programs.
The iTP3 project successfully achieved its goals and objectives in working with cohort one grantees to support and enable the development of innovative teen pregnancy prevention programs. By working closely with iTP3, innovative teams identified challenges and additional priority areas for subsequent years of the project. After work began with cohort one, the iTP3 team set priorities for selecting additional innovative teams and identifying types of capacity building and technical assistance needed with this kind of project. The team identified a need to shift capacity building, professional training, and innovation development as this project continued. Nonetheless, the activities completed with cohort one supported iTP3’s main goal―to support and enable innovation in teen pregnancy prevention.
The iTP3 project continued to expand capacity building opportunities with innovative teams and facilitated professional connections and referrals to provide unique opportunities for organizations and teams to enhance their program development by operating with a design thinking approach that can impact community-based systems. Allowing opportunities for people to utilize their creative and design thinking skills and to iterate solutions for their team is an essential core for the successful development of innovative programs. Cohort one of the iTP3 project brought together diverse professionals sharing an interest in innovative program development and provided working knowledge on how to improve the efforts to enable innovative programs.
This analysis of innovative program characteristics provides new insight into the development of innovative programs, both in general and within teen pregnancy prevention. To date, the majority of research on innovative health programs has focused on adoption and dissemination of health programs such as Rogers’ Diffusion of Innovation framework and Wandersman’s Interactive Systems Framework (Rogers, 1995; Wandersman et al., 2008) . The concept of providing mini grants with an opportunity and flexibility to iterate programs provides promise into new directions for the field. Additionally, there is promise in looking to other related fields for ideas on progressive program development. For example, the Socio Ecological Model, which recognizes intervening at various levels of intervention, has guided public health programming for nearly thirty years (McLeroy et al., 1988) . However, evidence-based teen pregnancy prevention programs still tend to focus at the individual level (Wilson et al., 2017) . Breaking from this individual-level paradigm provides a new way of thinking and approaching programs.
Most of the programs proposed during cohort one of the iTP3 project focused on innovation based on the setting or at the target population level. Through the course of working with the grantees there was a transition to innovation focusing on the design approach for program development and focusing on systems-level approaches.
6. Implications for Practice
The use of small or mini grants has been utilized in other public health efforts to aid in training and dissemination (McNeal, 2012; Kegler et al., 2015) . Small grants to support and enable innovation in teen pregnancy prevention programs provide an opportunity to explore the real challenges of program development in teen pregnancy prevention. iTP3 was charged with emphasizing innovation, but through the small grant efforts, the iTP3 team learned how to foster innovation through a flexible process and identify factors that make a program innovative. Additional exploration on whether small grants support innovation in program development and the process for developing innovative programs could provide meaningful insight. Given the current focus on program implementation and fidelity in the teen pregnancy prevention field, the impact of using design thinking in program development could also provide important insights.
Ultimately the larger project to support and enable innovation in teen pregnancy prevention provided the opportunity for the first cohort to propose ideas to expand the evidence-based programs in teen pregnancy prevention. The initial cohort proposal of innovation focused around geography and target population. Through the efforts of selecting programs, working with organizations, engaging with program teams, and evaluating the iTP3 capacity building support, the iTP3 project was successful in transitioning programs to further consider new ideas around program design and systems level approaches, beyond those initially proposed.
The project described was supported by Grant Number TP2AH000046 from the HHS Office of Adolescent Health. Contents are solely the responsibility of the authors and do not necessarily represent the official views of the Department of Health and Human Services or the Office of Adolescent Health.
Human Participant Compliance Statement
The Institutional Review Board at Texas A&M University approved all study protocols (Texas A&M University study number: 2016-0281).
 Douglas Jr., J. M., & Fenton, K. A. (2013). Understanding Sexual Health and Its Role in More Effective Prevention Programs. Public Health Reports, 128, 1-4.
 Douglas, K. (2007). Emerging Answers: Research Findings on Programs to Reduce Teen Pregnancy and Sexually Transmitted Diseases.
 Farb, A. F., & Margolis, A. L. (2016). The Teen Pregnancy Prevention Program (2010-2015): Synthesis of Impact Findings. American Journal of Public Health, 106, S9-S15.
 Fluhr, J. D., Oman, R. F., Allen, J. R., Lanphier, M. G., & McLeroy, K. R. (2004). A Collaborative Approach to Program Evaluation of Community-Based Teen Pregnancy Prevention Projects. Health Promotion Practice, 5, 127-137.
 Goesling, B., Colman, S., Trenholm, C., Terzian, M., & Moore, K. (2014). Programs to Reduce Teen Pregnancy, Sexually Transmitted Infections, and Associated Sexual Risk Behaviors: A Systematic Review. Journal of Adolescent Health, 54, 499-507.
 Kappeler, E. M., & Farb, A. F. (2014). Historical Context for the Creation of the Office of Adolescent Health and the Teen Pregnancy Prevention Program. Journal of Adolescent Health, 54, S3.
 Kegler, M. C., Carvalho, M. L., Ory, M., Kellstedt, D., Friedman, D. B., McCracken, J. L., Dawson, G., & Fernandez, M. (2015). Use of Mini-Grant to Disseminate Evidence-Based Interventions for Cancer Prevention and Control. Journal of Public Health Management and Practice, 21, 487-495.
 Maness, S. B., & Buhi, E. R. (2016). Associations between Social Determinants of Health and Pregnancy among Young People: A Systematic Review of Research Published during the Past 25 Years. Public Health Reports, 131, 86-99.
 Margolis, A. L., & Roper, A. Y. (2014). Practical Experience from the Office of Adolescent Health’s Large Scale Implementation of an Evidence-Based Teen Pregnancy Prevention Program. Journal of Adolescent Health, 54, S10-S14.
 McLeroy, K. R., Bibeau, D., Steckler, A., & Glanz, K. (1988). An Ecological Perspective on Health Promotion Programs. Health Education Quarterly, 15, 351-377.
 McNeal, M. S. (2012). Using Mini-Grants and Service-Learning Projects to Prepare Students to Serve Underserved Populations. Journal of Health Care for the Poor and Underserved, 23, 20-26.
 OAH (US Department of Health and Human Services, Office of the Assistant Secretary for Health, Office of Adolescent Health) (2015). Supporting and Enabling Early Innovation to Advance Adolescent Health and Prevent Teen Pregnancy (Tier 2A). DHHS Announcement No. AH-TP2 15-001, Washington DC: US Government Printing Office.
 OAH (US Department of Health and Human Services, Office of the Assistant Secretary for Health, Office of Adolescent Health) (2016a). Evidence-Based Teen Pregnancy Prevention Programs at a Glance.
 OAH (US Department of Health and Human Services, Office of the Assistant Secretary for Health, Office of Adolescent Health) (2016b). TPP Resource Center: Evidence-Based Programs.
 Penman-Aguilar, A., Carter, M., Snead, M. C., & Kourtis, A. P. (2013). Socioeconomic Disadvantage as a Social Determinant of Teen Childbearing in the U.S. Public Health Reports, 128, 5-22.
 Vechakul, J., Shrimali, B., & Sandhu, J. (2015). Human-Centered Design as an Approach for Place-Based Innovation in Public Health: A Case Study from Oakland, California. Maternal & Child Health Journal, 19, 2552-2559.
 Wandersman, A., Duffy, J., Flaspohler, P., Noonan, R., Lubell, K., Stillman, L., Saul, J. et al. (2008). Bridging the Gap between Prevention Research and Practice: The Interactive Systems Framework for Dissemination and Implementation. American Journal of Community Psychology, 41, 171-181.
 Wilson, K., Garney, W., Hays, C., Nelon, J., Farmer, J., & McLeroy, K. (2017). Encouraging Innovation in Teen Pregnancy Prevention Programs. Creative Education, 8, 294-303.