A Comparison Study of Onward to Opportunity (O2O) Utilizing a Matched Comparison Group from The Veterans Metrics Initiative (TVMI)
Onward to Opportunity (O2O) is a free career training program that provides professional certification and employee support services to transitioning service members, veterans and military spouses. O2O combines industry-validated curricula, strong partnerships with leading veteran service organizations and private sector companies, and comprehensive career coaching services to prepare and match you with your next career.
The Veterans Metrics Initiative (TVMI) is a longitudinal research study designed to assess the wellbeing of veterans after transitioning out of the military, including the identification of transition and reintegration programs accessed by the service member during transition. The study was conducted from 2016-2019 by researchers at Penn State’s Clearinghouse for Military Family Readiness. You can visit their website for more information on the Clearinghouse for Military Family Readiness.
Executive Summary
Career training and employment programs for servicemembers transitioning out of the military can be both over- and under-whelming. Many programs promise major results – often for a fee or use of earned benefits – but cannot substantiate these claims.
‘Proving’ a program’s efficacy requires specialized expertise and is often cost-prohibitive, which is why few career training programs – none in the transition space – are able to do so. In 2021 IVMF commissioned a third-party impact evaluation of Onward to Opportunity, a landmark study in the transition arena.
Evaluators from Penn State’s Clearinghouse for Military Family Readiness conducted a rigorous quasi-experimental study utilizing propensity score matching (PSM) to compare survey response data from The Veterans Metrics Initiative (TVMI) with O2O participant data. Their analysis found that O2O has statistically validated positive impacts on post-service employment, including starting salary, and characteristics of 6-month job retention.
In the two papers outlined below, the Clearinghouse team detail their methodology and key findings. This evaluation was completed with financial support from the Schultz Family Foundation.
Key Findings
O2O found to have “significant and positive” impact on transitioning service members
As stated above, the primary objective of this evaluation was to prove the value of O2O as an employment program for transitioning service members. We were not only able to accomplish this, but did so with a high level of statistical power, meaning the analysis was strong and the conclusions significant.
Onward to Opportunity is now the only veteran career training program that can demonstrate third-party validated efficacy for program participants with this level of evaluation rigor (i.e., experimental design).
Clearinghouse evaluators analyzed two primary employment outcomes: starting salary and an element of job retention – leaving a job for a better opportunity. Their findings definitively link program participation with higher salaries – even when participants do not fully complete the program. Also of note, findings were even stronger for the Jr. Enlisted population, who often face increased struggles post-transition.
O2O participants were also shown to be twice as likely to leave jobs at the 6-month mark for better opportunities than the TVMI control group. To us, this demonstrates the ability of the program to instill a strong career drive moving into the civilian sector, and an adeptness in identifying and capitalizing on career advancement opportunity.
See below for more detail on these findings and evaluators’ methodology.
Onward to Opportunity (O2O) and Effective Employment Components: A Comparison Study Utilizing Data from The Veterans Metrics Initiative (TVMI)
The first paper in this series lays the groundwork for the impact evaluation by defining the program components of Onward to Opportunity and aligning them with the components studied in The Veterans Metrics Initiative project.
In order to use the TVMI dataset as a valid control group, Penn State evaluators and IVMF staff needed to ensure that we could compare elements of O2O (e.g., resume writing, interview coaching, licensing/certification, job training, etc.) with the range of program components defined in TVMI. This ensured that we could compare “apples to apples” not only from a participant perspective, but also the programming perspective.
See the tables below for a list and description of the program components that were identified and mapped to components in TVMI.
Onward to Opportunity (O2O) and Effective Employment Components
Component | O2O Application |
Content and Process Components | |
Interviewing
| Installation teams and IVMF advisors execute individual learning plans and, with HHUSA, provide interview assistance or interview matching when individuals are ready. |
Resume writing
| Individual learning plans include personalized one-on-one resume writing assistance with a coach or mentor. O2O/HHUSA may also provide resume templates. |
Career planning, exploration
| This is done via mentors/coaches and networking groups and includes access to career coaching services, networking events, and workshops. The goal is for participants to learn personal marketing techniques and skills. |
Job training and certification
| This includes access to training and professional certifications and incorporates three learning pathways to further tailor training to participants’ industry of choice. |
| Barrier Reduction Components | |
| No transportation needed | Programs are offered on installations, which reduces the need for transportation. There is also a distance-learning online option. |
| Educational scholarships | The coursework is free and accessible for up to 6 months after participants’ completion of O2O. |
| Licensing assistance | One industry certification per person is funded after coursework and practice exams are completed. |
| Job placement | Individualized career services are provided via program partners – HHUSA. |
| Sustainability Components | |
| Referrals | Referrals to relevant resources are provided, and warm handoffs occur between O2O and HHUSA. |
| Alumni organization | After completion of program requirements, recipients are granted access to a nation-wide network of O2O alumni. |
A Comparison Study of Onward to Opportunity (O2O) Utilizing a Matched Comparison Group from The Veterans Metrics Initiative (TVMI): Sampling Frame and Outcome Analysis
The second paper in the series details the matched group comparison process and methods of analysis used to assess the impact of O2O, as well as the evaluation findings (summarized above).
First, the IVMF and Clearinghouse teams worked to “harmonize the data and specify the sampling frame,” which involved understanding the data collection strategies for both O2O and TVMI and making decisions on how to align the two datasets in cases of inconsistency. This helped evaluators create the matched sample between the two data sources, the basis for evaluation. The data were matched on multiple demographic characteristics, including age, gender, race/ethnicity, pay grade, service branch, and level of education. Of note, the O2O and TVMI samples used in this study were drawn from participants in 2017-2019 due to the potentially confounding impact of the COVID-19 pandemic.
Any O2O (Intent-to-Treat) Against TVMI Matched Sample Description
| Intervention Group: | Comparison Group: |
Age | 33.92 (SD=9.07) | 33.65 (SD=9.8) |
Male (Gender) | 79.6% | 80.3% |
Race/Ethnicity White Only NH | 55.2% | 57.1% |
Black Only | 16.6% | 15.4% |
Hispanic (Any Race) | 17.9% | 17.2% |
Asian HPI, Other Race NH | 12.2% | 10.3% |
Paygrade E1 to E3 | 3.4% | 1.9% |
E4 | 25.1% | 27.3% |
E5 | 22.3% | 21.6% |
E6 | 14.7% | 12.5% |
E7 to E9 | 12.2% | 10.7% |
O1 to O3 | 7.2% | 10.3% |
O4 to O7 | 14.4% | 15.0% |
Next, the Clearinghouse team evaluated Onward to Opportunity program outcomes using various statistical methods (for full descriptions, see full paper linked below). Analysis showed that servicemembers who participated in O2O, regardless of whether or not they were able to complete their learning pathway or certification exam, had significantly higher salaries than those in the TVMI matched sample.
Furthermore, those whose paygrade was E6 or lower and who completed the program had significantly higher salaries – $13,000 higher, on average.
Highest average starting salaries, by learning pathway.
Evaluators were also able to assess one element of job retention – leaving a job for a better opportunity. Specifically, if participants left their post-program job at or before the 6-month mark, did they do so for positive reasons? Analysis showed that O2O participants were twice as likely than the TVMI sample to leave a job for a better opportunity.
Program Updates & Considerations
The Clearinghouse team concluded the evaluation with a series of recommendations on steps the program could take to enhance future evaluation efforts and track additional program outcomes. Since participant data used for this evaluation was pre-2019 (per earlier note on COVID-19 having a potentially confounding impact on later data) some of the recommendations have already been addressed in recent data collection improvements.
Additionally, O2O has made several key program improvements that enhance the benefits demonstrated in this evaluation:
- Fully digitized Onward to Your Career content, now available to all participants (in-classroom and online)
- Expanded recruitment efforts and tailored program offerings for junior enlisted populations
- Virtual and extended reality tools for participants to simulate civilian job interviews (Learn more here)
- Added 20 new additional learning pathways (50+ total)
It is also important to remember that O2O always has – and continues to – benefit military spouses, a population that was not included in this evaluation. We were unable to include spouses in this project because the TVMI dataset was limited to service members and veterans, but we are confident that the benefits demonstrated in this evaluation extend to all populations of participants.
We aim to build upon this evidence base for continuous quality improvement through expanded program offerings, enhanced learner experience, increased completion, and employment outcomes.
Learn more about the Onward to Opportunity program and instructions for application.
