Onward to Opportunity (O2O) Impact Evaluation

A Comparison Study of Onward to Opportunity (O2O) Utilizing a Matched Comparison Group from The Veterans Metrics Initiative (TVMI)

Onward to Opportunity

Onward to Opportunity (O2O) is a free career training program that provides professional certification and employee support services to transitioning service members, veterans and military spouses. O2O combines industry-validated curricula, strong partnerships with leading veteran service organizations and private sector companies, and comprehensive career coaching services to prepare and match you with your next career.

Learn More

The Veterans Metrics Initiative

The Veterans Metrics Initiative (TVMI) is a longitudinal research study designed to assess the wellbeing of veterans after transitioning out of the military, including the identification of transition and reintegration programs accessed by the service member during transition. The study was conducted from 2016-2019 by researchers at Penn State’s Clearinghouse for Military Family Readiness.

Learn More

Executive Summary

Career training and employment programs for servicemembers transitioning out of the military can be both over- and under-whelming. Many programs promise major results – often for a fee or use of earned benefits – but cannot substantiate these claims.

‘Proving’ a program’s efficacy requires specialized expertise and is often cost-prohibitive, which is why few career training programs – none in the transition space – are able to do so. In 2021 IVMF commissioned a third-party impact evaluation of Onward to Opportunity, a landmark study in the transition arena.

Evaluators from Penn State’s Clearinghouse for Military Family Readiness conducted a rigorous quasi-experimental study utilizing propensity score matching (PSM) to compare survey response data from The Veterans Metrics Initiative (TVMI) with O2O participant data. Their analysis found that O2O has statistically validated positive impacts on post-service employment, including starting salary, and characteristics of 6-month job retention.

In the two papers outlined below, the Clearinghouse team detail their methodology and key findings. This evaluation was completed with financial support from the Schultz Family Foundation.

Program Components Study Impact/Comparison Study

Participants are 2x more likely to leave jobs for better opportunities at the 6-month mark

On average, O2O participants receive a $7,000 higher starting salary

On average, O2O participants receive a $13,000 higher salary for E-6 paygrade and below

As stated above, the primary objective of this evaluation was to prove the value of O2O as an employment program for transitioning service members. We were not only able to accomplish this, but did so with a high level of statistical power, meaning the analysis was strong and the conclusions significant.

Onward to Opportunity is now the only veteran career training program that can demonstrate third-party validated efficacy for program participants with this level of evaluation rigor (i.e., experimental design).

Clearinghouse evaluators analyzed two primary employment outcomes: starting salary and an element of job retention – leaving a job for a better opportunity. Their findings definitively link program participation with higher salaries – even when participants do not fully complete the program. Also of note, findings were even stronger for the Jr. Enlisted population, who often face increased struggles post-transition.

O2O participants were also shown to be twice as likely to leave jobs at the 6-month mark for better opportunities than the TVMI control group. To us, this demonstrates the ability of the program to instill a strong career drive moving into the civilian sector, and an adeptness in identifying and capitalizing on career advancement opportunity.

Program Components Study Impact/Comparison Study

The first paper in this series lays the groundwork for the impact evaluation by defining the program components of Onward to Opportunity and aligning them with the components studied in The Veterans Metrics Initiative project.

In order to use the TVMI dataset as a valid control group, Penn State evaluators and IVMF staff needed to ensure that we could compare elements of O2O (e.g., resume writing, interview coaching, licensing/certification, job training, etc.) with the range of program components defined in TVMI. This ensured that we could compare “apples to apples” not only from a participant perspective, but also the programming perspective.

See the tables below for a list and description of the program components that were identified and mapped to components in TVMI.

Component	O2O Application
Content and Process Components
Interviewing Rehearsal/Role-Play/Practice Mentor/Coach	Installation teams and IVMF advisors execute individual learning plans and, with HHUSA, provide interview assistance or interview matching when individuals are ready.
Resume writing Mentor/Coach	Individual learning plans include personalized one-on-one resume writing assistance with a coach or mentor. O2O/HHUSA may also provide resume templates.
Career planning, exploration Mentor/Coach Networking Group	This is done via mentors/coaches and networking groups and includes access to career coaching services, networking events, and workshops. The goal is for participants to learn personal marketing techniques and skills.
Job training and certification Direct Instruction	This includes access to training and professional certifications and incorporates three learning pathways to further tailor training to participants’ industry of choice.
Barrier Reduction Components
No transportation needed	Programs are offered on installations, which reduces the need for transportation. There is also a distance-learning online option.
Educational scholarships	The coursework is free and accessible for up to 6 months after participants’ completion of O2O.
Licensing assistance	One industry certification per person is funded after coursework and practice exams are completed.
Job placement	Individualized career services are provided via program partners – HHUSA.
Sustainability Components
Referrals	Referrals to relevant resources are provided, and warm handoffs occur between O2O and HHUSA.
Alumni organization	After completion of program requirements, recipients are granted access to a nation-wide network of O2O alumni.

Program Components Study

The second paper in the series details the matched group comparison process and methods of analysis used to assess the impact of O2O, as well as the evaluation findings (summarized above).

First, the IVMF and Clearinghouse teams worked to “harmonize the data and specify the sampling frame,” which involved understanding the data collection strategies for both O2O and TVMI and making decisions on how to align the two datasets in cases of inconsistency. This helped evaluators create the matched sample between the two data sources, the basis for evaluation. The data were matched on multiple demographic characteristics, including age, gender, race/ethnicity, pay grade, service branch, and level of education. Of note, the O2O and TVMI samples used in this study were drawn from participants in 2017-2019 due to the potentially confounding impact of the COVID-19 pandemic.

Next, the Clearinghouse team evaluated Onward to Opportunity program outcomes using various statistical methods (for full descriptions, see full paper linked below). Analysis showed that servicemembers who participated in O2O, regardless of whether or not they were able to complete their learning pathway or certification exam, had significantly higher salaries than those in the TVMI matched sample.

Furthermore, those whose paygrade was E6 or lower and who completed the program had significantly higher salaries – $13,000 higher, on average.

$ 74,821

Certified Information Systems Security Professional (CISSP)

$ 73,678

Project Management (PMP)

$ 65,811

Six Sigma Green Belt (SSGB)

Any O2O (Intent-to-Treat) Against TVMI Matched Sample Description

Intervention Group: Any O2O Program Use (n=319)	Comparison Group: TVMI (No Employment Program Use) (n=319)
Age	33.92 (SD=9.07)
Male (Gender)	79.6%
Race/Ethnicity White Only NH	55.2%
Black Only NH	16.6%
Hispanic (Any Race)	17.9%
Asian HPI, Other Race NH	12.2%
Paygrade E1 to E3	3.4%
E4	25.1%
E5	22.3%
E6	14.7%
E7 to E9	12.2%
O1 to O3	7.2%
O4 to O7	14.4%

Impact/Comparison Study

The Clearinghouse team concluded the evaluation with a series of recommendations on steps the program could take to enhance future evaluation efforts and track additional program outcomes. Since participant data used for this evaluation was pre-2019 (per earlier note on COVID-19 having a potentially confounding impact on later data) some of the recommendations have already been addressed in recent data collection improvements.

Additionally, O2O has made several key program improvements that enhance the benefits demonstrated in this evaluation:

Fully digitized Onward to Your Career content, now available to all participants (in-classroom and online)
Expanded recruitment efforts and tailored program offerings for junior enlisted populations
Virtual and extended reality tools for participants to simulate civilian job interviews (Learn more here)
Added 20 new additional learning pathways (40+ total)

It is also important to remember that O2O always has – and continues to – benefit military spouses, a population that was not included in this evaluation. We were unable to include spouses in this project because the TVMI dataset was limited to service members and veterans, but we are confident that the benefits demonstrated in this evaluation extend to all populations of participants.

We aim to build upon this evidence base for continuous quality improvement through expanded program offerings, enhanced learner experience, increased completion, and employment outcomes.

Program Components Study Impact/Comparison Study

Orange Alert

Onward to Opportunity (O2O) Impact Evaluation

A Comparison Study of Onward to Opportunity (O2O) Utilizing a Matched Comparison Group from The Veterans Metrics Initiative (TVMI)