Les formations professionnelles financées par l’État sont efficaces lorsqu’elles s’alignent sur les besoins des employeurs
La reprise économique postpandémie dépend fortement de la capacité des milliers de Canadiens licenciés de retrouver du travail. Le budget fédéral de 2021 a alloué des fonds supplémentaires à la formation axée sur les compétences et à l’aide à l’emploi pour les travailleurs les plus durement touchés par les mesures de confinement. Mais l’efficacité des formations financées par l’État à l’intention des sans-emploi et des travailleurs sous-employés suscite encore beaucoup de scepticisme.
Selon la présente étude de Karen Myers, Simon Harding et Kelly Pasolli, les doutes entourant l’utilité du financement public de la formation axée sur les compétences reposent sur des perceptions dépassées, issues d’anciennes évaluations sur des programmes de formation à grande échelle, dont la méthodologie est aujourd’hui contestée. L’analyse de 30 ans de données américaines et étrangères a permis aux auteurs de cerner les problèmes soulevés par ces évaluations passées et de faire ressortir les conclusions plus nuancées d’études menées depuis 10 ans sur les types de formation les plus utiles dépendant des situations et des participants. Selon le résultat clé de cette analyse, les formations financées par l’État sont surtout efficaces lorsqu’elles s’alignent sur les besoins des employeurs et les compétences demandées sur les marchés du travail locaux.
Deux modèles de formation centrés sur la demande, largement appliqués aux États-Unis, sont particulièrement prometteurs : la formation sectorielle et l’approche axée sur le cheminement de carrière. En analysant leurs effets sur les perspectives et les revenus d’emploi des participants, Karen Myers et ses coauteurs dégagent plusieurs facteurs de réussite : étroite collaboration avec les employeurs pour cibler les compétences en demande, minutieuse sélection des candidats selon les secteurs qui les intéressent, flexibilité dans la prestation des programmes de formation, et soutien complémentaire (comme les soins aux enfants et les conseils de carrière) pour contrer les obstacles à la formation rencontrés par les adultes en âge de travailler.
Les auteurs concluent que ces deux modèles (formation sectorielle et approche axée sur le cheminement de carrière) pourraient jouer au Canada un rôle considérable dans les politiques du marché du travail adoptées en réponse à la pandémie. Leur mise à profit optimale présente toutefois des défis pour les décideurs, chercheurs et intervenants. Pour mieux aligner nos systèmes de développement des compétences sur les besoins des employeurs et l’évolution du marché du travail, les auteurs recommandent aux décideurs d’examiner de près l’applicabilité de ces modèles à la situation canadienne, de tester et d’accroître l’utilisation des plus prometteurs, et de s’engager à apprendre des meilleures pratiques. Il sera aussi important de créer une solide infrastructure de réseautage entre employeurs et prestataires de formation, tout en produisant des informations actualisées sur le marché du travail.
Ces recommandations générales devraient servir de point de départ au vaste débat que les intervenants concernés ont déjà trop tardé à mener sur les moyens de rendre nos systèmes de développement des compétences plus souples et réactifs à l’évolution constante des besoins en compétences des travailleurs et des employeurs.
The economic impact of the COVID-19 pandemic has heightened the need for skills development in Canada to help adult learners navigate the rapidly evolving world of work.[1] Even before the pandemic, Canada’s skills development systems were failing to keep pace with changing labour market needs. Technological and demographic changes have increased the pressure on governments to offer more training and other support to workers — especially those with low skills — to prepare them for in-demand jobs, and to give employers access to the talent they require to thrive. These challenges call for governments to adopt more flexible, responsive and resilient workforce development approaches. How can that be achieved?
The first step in modernizing Canada’s skills development systems is to move away from the view that publicly funded training for unemployed or underemployed workers is ineffective and not cost-efficient. That outdated perception is based on past evaluations of US training programs whose methodology has been questioned. Instead, policy-makers need to better understand what types of training work and why, and then adopt successful models.
In this study, we draw on over 30 years of evidence from a range of countries to identify what approaches to skills development training are most effective and provide policy recommendations to lay the groundwork for replicating these models in Canada. We base our recommendations on evaluations of publicly funded training programs that focus specifically on outcomes for participants, such as getting a job and earning more.[2]
High-profile research conducted in the US from the 1980s onward to evaluate whether skills training works concluded that publicly funded training had, for the most part, a limited impact on participants’ employment or earnings outcomes, casting serious doubt on the value of investment in skills training. However, most of this research was based on large-scale skills training programs and was premised on a very general question: does any skills training produce returns for the average individual? More recent research led by practitioners, beginning in the 2010s, provides more precise and detailed investigation of questions such as, What sort of government-funded skills training works? Under what conditions does it work? For whom does it work best? They found that training can be effective, and that it is most effective when it aligns with employers’ needs and teaches skills that are in demand in local labour markets. The challenge for Canada’s policy-makers, therefore, is to learn from best practices here and abroad and reform our publicly funded training systems to ensure that skills development better responds to the evolving needs of employers and labour markets.
We begin by providing context on Canada’s skills development systems, stakeholders’ needs and the existing gaps and pressures that are driving the need for change. We then present the findings from our review of the evidence and describe some of the methodological challenges involved in evaluating training programs — especially large-scale publicly funded ones — that make it difficult to assess their effectiveness. We go on to describe two promising skills development approaches — sector-based models and the Career Pathways model — and explore the lessons learned in implementing these types of programs to guide experimentation with demand-driven approaches in Canada. In concluding, we propose a series of steps policy-makers could take to integrate these training models into how skills development works in Canada.
Our publicly funded skills training systems are built on two pillars. The first is K-12 and post-secondary education, which prepares young people to enter the labour market by delivering traditional academic and technical training. The second — which is the focus of this study — consists of active labour market programs, which provide unemployed or underemployed workers with various programs and services such as skills training and job-search assistance (Canada 2017a).
The provinces have jurisdiction over education (there are separate federal government arrangements for Indigenous education), while jurisdiction over labour market programs is more complex and has shifted over time. In the early 1990s, the federal government was the primary player in these programs, offering employment supports and training through its network of Canada Employment Centres. Then, in the mid-1990s, the federal government began devolving responsibility for the delivery of labour market programs to provinces and territories through the Labour Market Development Agreements, with the federal government funding the programs and the provinces and territories designing and delivering them. It took 15 years (until 2010) for all 13 jurisdictions to sign these bilateral agreements.
Although this devolution was intended to give provincial and territorial governments the flexibility to match programming to local conditions, we don’t know much about the extent to which flexibility increased or its impact, because there is little research on the subject. Now, as governments begin to plan for the postpandemic labour market recovery, knowledge about this issue is particularly relevant.
Moreover, although devolution has considerably decentralized active labour market programs, there has been little change in pan-Canadian arrangements to ensure coordination and mutual learning between governments (Wood 2016).
While Canada is an OECD leader in K-12 and post-secondary education, some experts argue (for example, Munro 2019) that it has underperformed for adult learners. On average, working Canadians receive only 49 hours of informal, job-related training per year, 9 hours fewer than the OECD average. Canada spends only 0.07 percent of its GDP on training, compared with the OECD average of 0.13 percent. And the workers who are most likely to need more training — those in rural and remote locations, those without post-secondary qualifications, and those with lower literacy — are also less likely to participate in training opportunities. To better meet adult learners’ needs, Canada’s labour market and skills programs need to be systematized and better coordinated (Munro 2019).
Another important shortcoming of Canada’s skills training systems is a lack of support for adults who wish to continue to improve their labour market prospects throughout their working lives. Currently, the programs focus primarily on assisting unemployed workers to obtain employment quickly, rather than enabling them to build the skills necessary to connect them to sustainable career paths and to navigate job and career transitions as labour markets evolve. University programs are also misaligned with adult learners’ needs, failing to equip them with the skills required to be employed in growth sectors (Canada 2017a).
The lack of evidence on the effectiveness of skills training, coupled with low participation levels and a complex funding environment, contributes to the perception that skills training offers little to workers (Jones 2011). Working Canadians need more agile, responsive skills development solutions.
Most of the existing evidence on the effectiveness of skills training comes from evaluations of publicly funded programs in the US over the last four decades. There is also some evidence — although it is not as rigorous — from Canada and Europe.
Before delving into details, let us note that our review paints a rather nuanced picture. Several high-profile, large-scale US evaluations of training programs in the 1980s concluded that the programs yielded negligible results. Those are the leading examples in a body of research that is responsible for much of the skepticism as to the value of training. Based on our review, we see two compelling reasons to question that conclusion:
The two large-scale evaluations of skills training programs that we present here are examples of the US research that concluded that skills training had limited effects on participant employment and earnings. They were conducted in 1980s and in the late 1990s using randomized controlled trials (RCTs).[3]
Large-scale evaluations of labour market programs began in the 1980s under the Job Training and Partnership Act. The highest-profile of these were the Dislocated Worker Demonstration Projects, which delivered skills training to displaced workers in four states between 1980 and 1987. The projects were designed to estimate the impact of skills training (classroom and on-the-job) on the employment and earnings outcomes of displaced workers relative to standard job-search assistance services. Although the outcomes for some workers (those younger than 45, women and those with “white collar” skills) were promising, the evaluation concluded that the skills training was, overall, no more effective than were standard job-search services (Leigh 1990).
In 1998, the Workforce Innovation Act replaced the Job Training and Partnership Act. The largest-scale evaluation carried out under the new act was that of the Adult and Dislocated Worker Programs, a suite of interventions that served about 7 million people. Local workforce areas were randomly assigned to three groups, which provided clients with one of the following sets of services:
The evaluation, which estimated the impact of receiving these sets of services on participants’ earnings, concluded that participating in training produced negligible results. Moreover, any differences in earnings between the groups that were offered training and those that were not were no longer evident two years after receiving services (Fortson et al. 2017).
These are the evaluations that cast doubt on spending government dollars on skills training for job seekers. However, in 1999 flaws were discovered in the methods used to evaluate the Job Training and Partnership Act programs (see Heckman, Lalonde and Smith 1999) that make it difficult to draw firm conclusions from their findings. In those RCTs, only some of the participants assigned to training (the treatment group) actually enrolled in and completed the training. As well, many control group participants sought training services elsewhere. If a large percentage of trial participants do not stick to the prescribed training regime, then a comparison of their outcomes leads to an underestimation of the impact of training.
Heckman and Smith (1998) highlighted the importance of this issue in their re-analysis of the data from the Job Training and Partnership Act demonstration projects. When they compared the outcomes of the participants in the treatment group who completed the training with those in the control group who did not access other training, they found that the effect of training was positive (though modest). They also reversed a finding from the earlier evaluation that had demonstrated that training had negative impacts on young males.
Similar methodological problems may have affected the evaluation of the Adult and Dislocated Worker Programs under the Workforce Innovation Act, because of particular differences among service providers, their training staff and the people receiving those services (Wilson 2019). For example, some service providers may not have offered the training due to lack of funding, and some clients may not have enrolled due to personal constraints. Furthermore, many individuals who received core and intensive services enrolled in training similar to that offered in full-service locations but funded by sources other than the Workforce Innovation Act.
While these large-scale evaluations under the Job Training and Partnership Act and the Workforce Innovation Act have received much attention due to their scope, a number of smaller US studies conducted between the 1960s and the 1990s also estimated the effectiveness of skills training. In 2003, Greenberg, Michalopoulos and Robins conducted a meta-analysis of these studies to obtain more accurate estimates of the effectiveness of training. Their analysis consisted of 31 evaluations of 15 US government-funded training programs for disadvantaged adults, including the Dislocated Worker Demonstration Projects. Two-thirds of these evaluations used experimental methods and one-third did not. They found that, overall, skills training increased earnings. And for women, the average annual earnings gains of around US$2,000, which appeared to persist several years after the training, is a notable outcome, given their relatively low pretraining incomes.
Other quasi-experimental studies have echoed these findings. Andersson et al. (2013) estimated that training provided to unemployed adults in two states (unnamed) resulted in average annual earnings gains of around US$1,200 and US$1,800 per person when compared with the earnings of those who received only other Workforce Innovation Act services.
Hollenbeck (2009) found that participation in training programs in Indiana funded through the Workforce Innovation Act increased the likelihood of employment by 10 percentage points for unemployed adults compared with participation in nontraining services. The quarterly increase in earnings was on average US$463 in the seventh quarter after completing training.
Jacobson, LaLonde and Sullivan (2002) estimated that participation in community college skills training in Washington State increased displaced workers’ earnings. Gaining at least one credit was associated with an increase in earnings of on average US$1,540 per year for men (9 percent) and US$880 per year for women (8 percent) over the long term (one year after leaving college).
To summarize, the US evidence shows that, while skills training may not be effective for all workers and in all contexts, there is some evidence it can improve some individuals’ labour market prospects.
There are several studies that looked at the effectiveness of skills training in Europe, across the OECD and beyond. Card, Kluve and Weber (2010) conducted a rigorous meta-analysis of 97 evaluations of labour market programs in 26 countries from 1995 to 2007. They found that a higher percentage of skills development programs resulted in increased earnings and employment for participants than did other types of programs. However, these positive effects only became apparent about two years after the participant had left the program.
Building on their earlier work, Card, Kluve and Weber (2017) carried out a meta-analysis of more than 200 evaluations (the majority of which used nonexperimental designs) of active labour market programs from around the world (half from Europe and one-quarter from non-OECD countries). They found that skills development programs had a greater overall effect on employment outcomes than did work-first interventions, such as job-search assistance, but that the full impact emerged only in the second year. In the first year, skills training increased a participant’s likelihood of getting a job by 3.9 percent; this increased to a 14 percent likelihood in the second year; and the increase was sustained in the third year (a 13.6 percent likelihood).
Kluve (2010) carried out a meta-analysis of 137 evaluations of active labour market programs in 19 European countries from the early 1980s to the mid-2000s. Overall, training programs were found to have a modest likelihood of boosting postprogram employment rates: out of 70 evaluations, 38 reported significant positive impacts.
In summary, this evidence shows that skills training can be effective for some participants in certain contexts, although impacts take time to fully emerge.
The evidence base in Canada is not as well developed as that in the US. Although numerous evaluations of Canadian skills training programs do exist, we do not consider them in this study, because they do not rigorously evaluate the impact of training for participants.
Nevertheless, the federal government regularly evaluates the Labour Market Development Agreements, which include publicly funded skills training for Employment Insurance (EI) recipients (also known as EI Part II). The most recent evaluation found that active claimants (i.e., those receiving EI payments) who participated in skills development interventions saw their earnings rise (Canada 2017b). Male participants gained on average, over three years, a total of C$8,500 and female participants gained C$6,600.[4] Participants in skills development interventions were also more likely to find jobs after completing training. While encouraging, these findings should be interpreted with caution, since the evaluation took into account only some of the factors that might explain the differences in outcomes between individuals who received training and those who did not.
At the provincial level, one of the few skills development programs that have been rigorously evaluated is Ontario’s Second Career program, which provides financial assistance to job seekers for classroom-based vocational training. This evaluation found mixed evidence regarding the effectiveness of skills training. Based on data from 2002-05 and 2007-08, active claimants who participated in training had, on average, a cumulative earnings gain of C$6,000 in the three-year period following the training, and an additional C$7,000 overall in years four and five following the training. For former claimants (i.e., those who had received but were no longer receiving EI benefits when they participated in the training), the corresponding results were C$2,800 in the first three years and C$5,500 in years four and five. However, the program had a high cost (approximately C$8,500 per participant), relatively lengthy payback periods (7.8 to 14.3 years), [5] and a negative cost-benefit ratio for former claimants (Canada 2017c).
While high-profile RCTs conducted in the US have reported mixed findings on the impact of publicly funded skills training, our review of the evidence highlights some methodological issues that bring into question the view that skills training is ineffective. We also found compelling evidence of skills training having positive impacts in the US and elsewhere. This leads us to the more nuanced conclusion that training can be effective under certain conditions.
Training practitioners have built on these insights to experiment with types of skills training models that aim to maximize their effectiveness for participants and employers. In the following section, we explore the evidence behind some of these models.
Since the early 1990s, two approaches have gained traction: the sector-based model and Career Pathways. The sector-based model aims to align training with in-demand occupations in the local economy by working with employers in specific industries to identify their skills needs and then to design training that accommodates those needs. While the evidence base for the efficacy of these models is still limited, rigorous evaluations of major sector-based programs in the US have reported encouraging results for participants. The Career Pathways model combines sector-based training with modular post-secondary credentials to help workers enter and make career progress in growth sectors.[6]
These models emerged in the US in the early 1990s as modest-scale projects to meet local labour market needs. They were mostly funded by private charitable foundations and addressed employers’ dissatisfaction with traditional approaches to training, which were seen as disconnected from their needs. These models were perceived as strengthening links with industry by tailoring training to the needs of existing jobs, so trainees would have a better chance of obtaining and retaining these jobs (Holzer 2017). Box 1 presents common features of sector-based training programs.
In the next section, we present two landmark large-scale US studies that highlighted the potential for sector-based training to improve employment opportunities and earnings: the Sectoral Employment Initiative Study and WorkAdvance Demonstrations. Both found that sector-based training benefited participants. They also identified important factors in successfully implementing sector-based training, in particular ensuring that training providers fully understand the target sector, local labour market dynamics and employers’ needs.
This study (2003-07) evaluated the following three sector-based training programs using an RCT design (Maguire et al. 2010):
All three training providers were committed to serving disadvantaged job seekers. About two-thirds of the participants were unemployed at the time of the study, and 37 percent had received public assistance at some point prior to training.
The employment and earnings outcomes for participants in sector-based training programs were compared with those for a control group that could access other services but did not participate in the sector-based training. During the two-year study period, training participants earned, on average, a total of US$4,500 more than did those in the control group.[7] Training participants at all three program sites made significant earnings gains, although the gains for those who participated in training at Jewish Vocational Service Boston and Per Scholas emerged only in the second year, indicating a “lock-in effect.”[8]
Over the same period, training participants were employed for an average of 1.3 months longer than were members of the control group. They also worked significantly more hours (around 250 more in each year of the study). In the second year, training participants were significantly more likely to work all 12 months of the year than were those in the control group, and they were more likely to be employed in jobs with higher wages and benefits (Maguire et al. 2010).
The success of the three programs evaluated in the Sectoral Employment Impact Study, compared with earlier skills training programs, inspired further refinement and testing of the sector-based approach. WorkAdvance (2011-15) built on the findings of the Sectoral Employment Impact Study by testing a more intensive sector-based training model that featured thorough screening, career readiness services, sector-based training for in-demand jobs, job development and placement, and retention and advancement services. WorkAdvance was implemented in a four-site demonstration by the following providers of sector-based training programs and evaluated using an RCT design:
The long-term evaluation, which was conducted over a two-year period (2017-18) between four and eight years after participants entered the study, found encouraging results. Participants at Per Scholas experienced an average annual wage increase of US$6,281 — an exceptionally large impact for a random assignment study — compared with outcomes for the control group. At the other sites, the earnings gains were smaller and not statistically significant. The earnings increases appeared to be driven by higher wages, since none of the programs resulted in significantly higher employment levels. The large earnings impact at Per Scholas may be related to the high rates of job entry into the target sector, facilitated by the organization’s strong relationships with local employers and industry. Training participants were also significantly more likely than those in the control group to be earning high salaries (over US$30,000): a difference of 6.3 percentage points at Per Scholas, 9.3 percentage points at St. Nicks Alliance and 5.8 percentage points at Madison Strategies Group (Schaberg and Greenberg 2020).
While the Sectoral Employment Impact Study and WorkAdvance evaluations showed that training programs overall improved participants’ employment and earnings outcomes, the effects varied across program sites. To understand the reasons for the variations, researchers examined the success factors and challenges in implementing effective sector-based models.
A takeaway from these evaluations is that sector-based models can be difficult to implement, even for high-capacity providers. Some providers of WorkAdvance training programs received considerable technical support, yet it took almost a year to implement all components of the training model, including recruitment and screening, training curriculum development, job placement and retention, and advancement services.
The studies also highlighted the importance of employers’ engagement in the design and delivery of effective sector-based programming. While many training providers had expertise in serving participants with specific needs, some had difficulty combining that expertise with an understanding of employers’ needs and thus delivering training aligned with those needs. The providers that produced the best results in these studies were those who offered training in sectors with strong local labour demand and successfully placed participants in jobs in that sector. Effective employer outreach proved important, as did working with employers to help them articulate their needs, getting their feedback on training content and encouraging them to engage with the program.
The Career Pathways model emerged from training practitioners’ experimentation in the US in the early 2000s. It brings an employer-focused approach to post-secondary education and training by combining traditional community college instruction with shorter-term training to prepare adults for in-demand jobs and requires less financial investment and time commitment than do traditional routes into these jobs (Holzer 2017). The Career Pathways model gives workers the skills required to transition into a new sector at entry level, and then it helps them to progress in that sector by providing regular upskilling opportunities. Carefully curated, linear, step-by-step “pathways” simplify and facilitate learners’ progress.
Two key features distinguish Career Pathways from sector-based training models. First, Career Pathways is delivered through the post-secondary education system. Second (and most importantly), participants can continue to engage with Career Pathways over a long period of time, which is built into the pathways approach.
The Career Pathways model has the following four broad features that reduce the barriers faced by working-age adults who want to participate in training:
While the specific pathways vary according to the needs of the sector, the participants involved, and other context-specific factors, they generally feature the steps shown in figure 1. Individuals progress from basic skills or bridge training to more advanced credentials. Each step is linked to an increasingly skilled and well-paying job (the demand for which is first established based on local labour market data). As participants progress along the pathway, they accumulate credentials that reflect the skills they have acquired along the way.
The Career Pathways model was initially driven by experimentation and partnerships between community workforce development organizations and community colleges. It is now widely used across the US as decision-makers have begun to adopt it as a promising approach to skills training. In 2014, the US federal government integrated a funding framework for the Career Pathways model into the Workforce Innovation and Opportunity Act.[10] A number of states, including California, Washington, Oregon and Kentucky, have developed their own Career Pathways frameworks, as have numerous cities, including New York, St. Louis and San Diego.
There is now an emerging evidence base on the effectiveness of Career Pathways programs. Below we review these findings and summarize the key takeaways from three large-scale evaluations in the US: Pathways for Advancing Careers and Education, Health Profession Opportunity Grants and Accelerating Opportunity.
This project (2007-18) evaluated nine promising Career Pathways programs using an RCT design. All programs featured an innovative approach to basic skills and occupational training, support services to address academic and nonacademic barriers to training, and connections to employers during and after the program (Gardiner and Juras 2019). Most of the programs were in the health care sector, although some targeted information technology and finance.
The evaluations compared training participants’ intermediate outcomes[11] with those of a control group that could access other locally available services but not Career Pathways. The intermediate outcomes objectives varied across programs, but all included either higher educational attainment or earnings gains. Results indicate that seven of the nine programs had a significant impact in terms of achieving these outcomes.
The two programs that did not improve outcomes were also found to have a limited impact on the use of support services; that is, the Career Pathways participants in these programs did not use services such as career counselling and job placement assistance to a greater extent than did control-group participants. This finding indicates the importance of these support services in explaining the success of Career Pathways programs.
These grants (2010-present) provide funding to organizations to deliver Career Pathways programs designed to help public assistance recipients transition into in-demand health care occupations. In the first cycle, 23 grantees delivered 42 programs that adhered to federal Career Pathways guidelines.[12] These programs are being evaluated using an RCT design.
At 30 months following enrolment, 75 percent of training program participants had made progress on educational attainment (e.g., training enrolment, receipt of financial assistance for tuition), compared with 63 percent of control-group participants (Peck et al. 2019). The evaluators also found that training participants were more likely to earn a college certificate or diploma, or a vocational training certificate.
The impact of these grants on participants’ likelihood of getting a job at 30 months was small (+1 percent) and there was no detectable impact on earnings. However, there is evidence that the programs improved the likelihood of participants finding employment in the target sector: after completing training, 56 percent of participants in the treatment group were working in health care, 12 percentage points higher than in the control group. The programs also had a small but significant positive impact on job characteristics such as having health insurance. Furthermore, 43 percent of the treatment group had made career progress (i.e., had completed training and had increased earnings) compared with 36 percent of the control group.
This Career Pathways program (2011-15) was directed at adults without a high school diploma. Training participants entered high school equivalency programs, which led to stackable (i.e., accumulated sequentially over time) industry-recognized credentials in a range of sectors. They received career support services, job-search assistance, job development and placement services.
The evaluation used a control group consisting of students not associated with Accelerating Opportunity who were matched on observable characteristics. The program had a large positive impact: training participants were 10 to 20 percentage points more likely to earn a credential relative to control group participants across all four states where the program was implemented.
At the same time, employment outcomes were mixed. Most Accelerating Opportunity participants were unable to translate their credentials into consistent employment and earnings gains over the two-year evaluation period, possibly due to weak relationships between training providers and employers.
The evaluation also found that accessing support services was positively linked to participants’ education and employment outcomes, but these services were costly and colleges often struggled to find sufficient funds to provide them.
Evaluations of Career Pathways programs have highlighted some important lessons for their successful implementation. As is true of other sector-based training, Career Pathways programs require deep engagement with employers in order to understand their skills needs and design appropriate training and credentials. Because many Career Pathways programs are based on close cooperation among post-secondary institutions, employers and community service organizations, they necessitate significant investment in partnership development and coordination.
Access to support services also plays an important role in the success of the Career Pathways programs that deliver the best results for participants in terms of employment and earnings outcomes. One explanation may be that these programs tend to serve individuals who face barriers to labour market entry or to re-training.
The consistently positive results from evaluations of sector-based training models highlight the importance of aligning training with the needs of employers and local labour markets. While the evidence on Career Pathways is mixed, some promising early results, as well as the widespread adoption of this model across the US, make it worth considering as a way to incorporate labour market demand in the design of post-secondary training programs.
Although the literature also shows that designing, implementing and testing sector-based and Career Pathways training models is complex, dealing with the complexity is key to unlocking the potential of these models to make skills development systems more effective and more responsive to the needs of participants.
Given what we know about skills training, what steps should we take to strengthen Canada’s skills development systems? While the evidence on the effectiveness of traditional publicly funded training programs is mixed, we can conclude that programs that are aligned with employers’ and labour market needs, such as those inspired by sector-based models, can be effective.
The need to adapt and adjust Canada’s skills development systems to labour market needs becomes more pressing given the potentially long-lasting consequences of the COVID-19 pandemic for workers and for the economy. The sector-based and Career Pathways training models hold considerable promise and could play an important role in Canada’s labour market response to the pandemic. However, determining how to leverage these models to their full potential in this new context is a major challenge facing policy-makers, researchers and training practitioners.
Below, we outline four steps policy-makers can take to better align Canada’s skills development systems with employers’ needs, in the context of changing labour markets. Where appropriate, we refer to Ontario’s ongoing efforts to implement these models.[13] We recognize, however, that the way these broad recommendations are implemented would differ from province to province depending on their particular training needs.
Sector-based training builds skills that are tied to a specific industry sector. Multiple rigorous evaluations of sector-based training programs have shown that this type of training improves participants’ employment and earnings outcomes. Policy-makers across Canada should invest in testing, replicating and scaling these types of programs to maximize workers’ opportunities to participate in training that is in line with current labour-market demand and can have a meaningful impact on their employment prospects.
Efforts to scale up effective sector-based models must include identification of good candidates for scaling. The focus should not be entirely on the outcomes for participants. Careful attention should also be paid to how the models are implemented, to account for factors such as demand for skills and value for money. It may be that the models that are most suitable for scaling are those that respond to Canada-wide skills shortages, rather than to specific local labour market situations.
The literature has shown that, while sector-based training programs are effective, launching and operating them can be complex. As policy-makers explore how to implement and scale these types of programs, they must ensure that training providers have the time and resources needed to establish strong partnerships with employers, understand labour market needs, and design high-quality training that aligns with those needs.[14] Building the capacity of a wide range of training providers will also be key to effectively scaling sector-based approaches.
Ontario has started to move in this direction. SkillsAdvance Ontario funds projects that offer job-seekers sector-specific employment and training services so employers can recruit and advance workers with the right skills.
Few other provinces or territories have institutionalized sector-based training as part of their employment and training systems. Nevertheless, all of them have community colleges and polytechnics that offer vocational training and that often have strong links with employers. While these post-secondary institutions would be key players in the design and delivery of Career Pathways programs, the vocational training courses they currently offer are not Career Pathways programs as they do not lay out a clearly delineated pathway for progress or package the training and credentialing to improve accessibility for working adults.
There are other examples of experimentation with sector-based training across Canada. For instance, the Future Skills Centre is a six-year initiative funded by the federal government in 2018 to experiment and generate evidence on effective skills development models. It has invested in multiple sector-focused pilot projects, including a rapid-response project to provide upskilling and retraining services to workers displaced from the hospitality and tourism sector as a result of the COVID-19 pandemic (Canada 2020a). Evaluating these projects to identify what works and scaling up the most promising approaches will help expand the reach and impact of sector-based training across Canada.
Ensuring that sector-based training programs are sustainable and responsive requires having the necessary policy infrastructure consisting of partnerships, agreements and governance arrangements. In considering the implementation of sector-based programs, policy-makers should also consider building system infrastructure that enables their effective design, implementation, sustainability and replication.
One critical piece of that infrastructure is timely, accurate information on labour market trends and skills needs across sectors to ensure that the training provided equips workers with the skills they need to succeed. The Labour Market Information Council, established in 2018, has developed several data dashboards that provide the public with information on skills needs and in-demand occupations. It is also working on improving the granularity of this information.
Skills mapping tools can also provide valuable information about the skills needed to succeed in particular sectors (see Bonen and Oschinski 2021). For example, the Future Skills Centre’s rapid response initiative for displaced workers in tourism and hospitality uses a skills mapping database (developed by the Conference Board of Canada) to identify sectors that use similar skills in order to provide opportunities for targeted retraining. While skills forecasting can also play an important role in the design of training programs, predicting future needs is extremely challenging, even on short time horizons (e.g., six months into the future), because it requires large amounts of reliable data and significant technical knowledge.
Skills mapping and demand forecasting must also take into account cyclical patterns in the labour market. Periodic economic downturns may have different impacts across industries. Similarly, political cycles will affect opportunities in sectors that rely on public funding (particularly in health care). Having a better understanding of how these factors affect the demand for skills will help policy-makers and practitioners find the right balance that will enable participants to get jobs in their target sector and acquire more general, transferable skills that would be useful should they wish to transition to another industry.
Sustainable sector-based models also rely on strong networks that connect employers, training providers and other workforce development organizations. Policy-makers can facilitate the development of these networks by providing information-sharing platforms and referrals, and by supporting organizations working together to design and implement sector-based training. The Ontario government has made investments in this area through Tourism SkillsNet Ontario, an initiative funded through SkillsAdvance Ontario that brings together stakeholders in the hospitality and tourism sector to address common skills needs.[15]
In addition to funding sector-based training programs, policy-makers should explore ways to more closely align post-secondary education programming for working-age adults with labour market needs. The Career Pathways model offers a framework for achieving this through its flexible, modular credentials geared toward in-demand job opportunities in local labour markets. Although community colleges — especially polytechnics — would undoubtedly play an important role in delivering Career Pathways programs in Canada, more work is needed to evaluate how the programs can be integrated into the existing training landscape.
In 2018, the Ontario government and the Ontario Centre for Workforce Innovation supported a Career Pathways pilot project at Conestoga College — a polytechnic — to test the feasibility of implementing the Career Pathways model in Ontario. The pilot project explored the successes, challenges and lessons learned from two programs that provide a first “step” in a career pathway. The programs focused on building the academic and workplace skills that learners needed for entry-level employment in social care and in warehousing and logistics while also providing a bridge to more advanced college credentials and employment opportunities.
Although Career Pathways is a promising framework, its large-scale implementation would require provinces and territories to make significant, complex changes to their skills development systems. They would have to take the following measures:
Based on our review of the evidence, the effectiveness of training programs is influenced by their design and by the contexts and conditions in which they are implemented. While sector-based and Career Pathways training models have produced some encouraging results in the United States, they are not a panacea. Policy-makers should invest in continuously generating evidence about what models work, for whom and under what conditions.
Our review also identifies some important gaps in our knowledge about the effectiveness of these models. For example, evaluations still do not fully investigate the long-term impact of Career Pathways on participants’ employment and earnings outcomes. Also, compared with the US, where the evidence base for these models is robust, there has been only limited experimentation and evaluation of these models in the Canadian context. Important questions remain about how to effectively adapt the Career Pathways model to ensure positive impacts for participants across Canada.
The federal government has demonstrated a commitment to learning about what works in skills development by creating the Future Skills Centre, which is mandated to experiment and generate evidence about emerging skills development approaches. In the short term, the task facing the centre is to develop collaborative skills training models that focus on sectors hit by the COVID-19 pandemic, something that it is pursuing as a case study with its rapid-response initiative to help displaced hospitality workers pivot to in-demand jobs that match their skills and expertise (Canada 2020b).
The need for timely and responsive skills development options has been thrust front and centre by the COVID-19 pandemic, which has created widespread economic disruption. Canada needs training that equips job seekers and workers with skills that are better aligned with labour market demand.
The evidence base for publicly funded skills training provides important lessons for policy-makers looking to respond to the economic impacts of COVID-19. Models such as sector-based training and Career Pathways hold promise for adapting our skills development systems to meet employers’ needs. The evidence also highlights the complexity involved in designing and delivering effective training programs, pointing to the need for investments in system infrastructure, and experimentation and evaluation to ensure that the programs are flexible, responsive and resilient. Such investments would help strengthen our skills development systems and produce better outcomes for individuals, communities and employers across Canada.
We offer four recommendations to support this transformation:
We hope that these broad recommendations can be starting points for substantive discussions among key stakeholders on how to transform Canada’s skills development systems to make them more flexible and responsive to evolving labour market needs.
[1] We use the terms “skills development” and “skills training” interchangeably.
[2] Another type of program evaluation assesses a program’s strategic alignment and implementation, rather than its effects on participants’ employment outcomes. Because those evaluations are not aligned with the goals of this study, we did not review them.
[3] RCTs (an experimental design) have an advantage over nonexperimental design: they can isolate the impact of training on employment and earnings from other factors that might also influence outcomes. Hence, experimental methods (and particularly RCTs) are the gold-standard approach to program evaluation.
[4] The extensive literature on gender differences in outcomes due to training is divided on whether women’s returns to training are lower than those for men, and it presents a wide range of explanations for the differences (see McKerrow et al. 2020). That topic is outside of the scope of this study.
[5] The payback period is the time required for the cumulative benefits from the training to equal the cost of providing it. We present a range of payback periods, because the types and cost of training varied, as did the types of post-training employment and the time it took to find jobs. For Ontario’s Second Career Program, the cost-benefit ratios were negative for former EI claimants, a group that includes the long-term unemployed.
[6] The Career Pathways model is distinct from micro-credentials in that it offers participants the opportunity to continuously engage in medium-to-long-term sector-based training to advance in their chosen sectors. Micro-credentials, which are abridged credentials aligned with labour market needs, do not need to be organized into a pathway. Given the need for more agile skills recognition, micro-credentials have the potential to play a major role in Career Pathways. There is an emerging market for them in the US (see Coté, White and Cuenco 2020) and they need to be evaluated. They are, however, beyond the scope of this study.
[7] Participants were observed over 24 months, starting after they met the eligibility criteria. Because participants spent the first 12 months of the study in training, internships and the initial job search, Maguire et al. (2010) evaluate the outcomes separately over the full 24-month study period and over months 13-24, which is when participants became available to participate in the labour market.
[8] The term “lock-in effect” refers to the delay in the emergence of positive outcomes for participants; these positive outcomes can appear months or years after the program is launched. The delay usually occurs because participants are enrolled in training and so are not actively looking for employment. The absence of this effect at Wisconsin Regional Training Partnership is likely due to that training provider’s emphasis on quick job placement.
[9] Per Scholas was initially included in the Sectoral Employment Impact Study; it was then enhanced for WorkAdvance, with additional support services for participants.
[10] United States. Workforce Innovation and Opportunity Act. 2014. Pub. L. No. 113-128.
[11] An intermediate outcome is an outcome that must occur in order for the participant to achieve a critical outcome. For example, entry into post-secondary education is an intermediate outcome that is needed if a participant is to obtain a post-secondary credential (critical outcome).
[12] The latest version of the Career Pathways Toolkit was developed in 2015 (see United States 2015b).
[13] The intention here is not to suggest that Ontario is the only province taking steps in this area, but rather to use the Ontario example to show how these steps could unfold at the provincial level.
[14] Defining and measuring the quality of training is beyond the scope of this study.
[15] Ontario’s tourism sector has been at the forefront of skills training in Canada. That industry has been particularly harmed nationwide by the COVID-19 pandemic.
[16] Wraparound supports deal with personal circumstances, or barriers, that might impede individuals’ ability to participate in training. These include mental and physical health issues, lack of essential skills such as literacy, and access to child care and transportation.
Andersson, F., H.J. Holzer, J.I. Lane, D. Rosenblum and J. Smith. 2013. Does Federally Funded Job Training Work? Nonexperimental Estimates of WIA Training Impacts Using Longitudinal Data on Workers and Firms (No. w19446). Cambridge, MA: National Bureau of Economic Research.
Bonen, T., and M. Oschinski. 2021. Mapping Canada’s Training Ecosystem: Much Needed and Long Overdue. Montreal: Institute for Research on Public Policy.
Canada. 2017a. Minister of Finance. Learning Nation: Equipping Canada’s Workforce with Skills for the Future. Advisory Council on Economic Growth.
——————. 2017b. Economic and Social Development Canada. Evaluation of the Labour Market Development Agreements: Synthesis Report, Strategic and Service Policy Branch.
——————. 2017c. Economic and Social Development Canada. Evaluation of the Canada-Ontario Labour Market Development Agreement: Synthesis Report, Strategic and Service Policy Branch.
Card, D., J. Kluve and A. Weber. 2010. “Active Labour Market Policy Evaluations: A Meta-
Analysis.” Economic Journal 120(548): F452-77.
Canada. 2017a. Minister of Finance. Learning Nation: Equipping Canada’s Workforce with Skills for the Future. Advisory Council on Economic Growth.
——————. 2017b. Economic and Social Development Canada. Evaluation of the Labour Market Development Agreements: Synthesis Report, Strategic and Service Policy Branch.
——————. 2017c. Economic and Social Development Canada. Evaluation of the Canada-Ontario Labour Market Development Agreement: Synthesis Report, Strategic and Service Policy Branch.
——————. 2020a. Future Skills Centre. “Future Skills Centre and OTEC Announce Rapid Response Project for Hard-Hit Tourism and Hospitality Workers.” https://fsc-ccf.ca/engage/future-skills-centre-and-otec-announce-rapid-response-project-for-hard-hit-tourism-and-hospitality-workers/
——————. 2020b. Future Skills Centre. “Rapid Response in Times of Disruption: Prototyping a Case Study in the Hard-Hit Hospitality Sector.” https://fsc-ccf.ca/projects/rapid-response-in-times-of-disruption/
——————. 2017. “What Works? A Meta-Analysis of Recent Active Labor Market Program Evaluations.” Journal of the European Economic Association 16(3), 894-931.
Coté, A., A. White and M. Cuenco. 2020. A Lifelong Learning Strategy for Ontario. Ontario 360, Policy paper.
Fortson, K., D. Rotz, P. Burkander, A. Mastri, P. Schochet, L. Rosenberg, S. McConnell and
R. D’Amico. 2017. Providing Public Workforce Services to Job Seekers: 30-Month Impact Findings on the WIA Adult and Dislocated Worker Programs. Mathematica Policy Research. Washington, DC: US Department of Labor.
Gardiner, K., and R. Juras. 2019. Pathways for Advancing Careers and Education (PACE): Cross-Program Implementation and Impact Study Findings. OPRE Report No. 2019-32.
Greenberg, D.H., C. Michalopoulos and P.K. Robins. 2003. “A Meta-Analysis of Government-Sponsored Training Programs.” ILR Review 57(1): 31-53.
Heckman J., R. Lalonde and J. Smith. 1999. “The Economics and Econometrics of Active Labor Market Programs.” In Handbook of Labor Economics, edited by O. Ashenfelter and D. Card. (Vol. 3, 1865-2097). Amsterdam: Elsevier.
Heckman J., and J. Smith. 1998. “The Sensitivity of Experimental Impact Estimates: Evidence From the National JTPA Study.” In Youth Employment and Unemployment in the OECD Countries, edited by R. Freeman and L. Katz. Chicago: University of Chicago Press.
Hollenbeck, K. 2009. Returns on Investment Analysis of a Selected Set of Workforce Systems Programs in Indiana. Kalamazoo, MI: W.E. Upjohn Institute for Employment Research.
Holzer, H. 2017. Sector-Based Training Strategies: The Challenges of Matching Workers and Their Skills to Well-Paying Jobs. McCourt School of Public Policy. Washington, DC: Georgetown
University.
Jacobson, L., R. LaLonde and D. Sullivan. 2002. Estimating the Returns to Community College Schooling for Displaced Workers. Washington, DC: US Department of Labor.
Jones, S. 2011. The Effectiveness of Training for Displaced Workers with Long Prior Job Tenure. Mowat Centre. Toronto: University of Toronto.
Kluve, J. 2010. “The Effectiveness of European Active Labor Market Programs.” Labour Economics 17(6): 904-18.
Leigh, D. 1990. Does Training Work for Displaced Workers? Kalamazoo, MI: W.E. Upjohn Institute for Employment Research.
Maguire, S., J. Freely, C. Clymer, M. Conway and D. Schwartz. 2010. Tuning in to Local Labor Markets: Findings from the Sectoral Employment Impact Study. Philadelphia: Public/Private Ventures.
McKerrow, M., S. Harding, N. Conte and M. Palamar. 2020. Skills Development and the Gender Wage Gap: An Analysis of Skills Training Programs in a Large Administrative Data Platform. Working paper. Employment and Social Development Canada.
Munro, D. 2019. Skills Training and Lifelong Learning. Brave New Work Key Issues Series. Ottawa: Public Policy Forum.
Peck, L., D. Litwok, D. Walton, E. Harvil and A. Werner. 2019. Health Profession Opportunity Grants (HPOG 1.0) Impact Study: Three-year Impacts Report. OPRE Report 2019-114. Washington, DC: US Department of Health and Human Services.
Schaberg, K., and D.H. Greenberg. 2020. Long-Term Effects of a Sectoral Advancement Strategy: Costs, Benefits, and Impacts from the WorkAdvance Demonstration. New York: Manpower Demonstration Research Corporation.
United States. 2015a. Department of Health and Human Services, Administration for Children and Families. “About Career Pathways.” https://career-pathways.org/about-career-pathways
——————. 2015b. Department of Labour. Career Pathways Toolkit: A Guide for System Development. https://wdr.doleta.gov/directives/attach/TEN/TEN_17-15_Attachment_Acc.pdf
Wilson, B. 2019. “Understanding the New Evaluation of WIA: It Doesn’t Say What You Might Think It Says.” Washington, DC: National Skills Coalition, January 14. https://www.nationalskillscoalition.org/blog/higher-education/understanding-the-new-evaluation-of-wia-it-doesnt-say-what-you-might-think-it-says/
Wood, D.E. 2016. Strengthening Canada’s Public Employment Service Post-Devolution. Ottawa: Caledon Institute of Social Policy.
This study was published as part of The Future of Skills and Adult Learning research program, under the direction of Natalia Mishagina. The manuscript was copy-edited by Elizabeth Macfie proofreading was by Zofia Laubitz, editorial coordination was by Francesca Worrall, production was by Chantal Létourneau and art direction was by Anne Tremblay.
Karen Myers is president and CEO of Blueprint, an organization that collaborates with leaders across Canada to solve complex social and economic issues. She is a leading voice for the use of evidence and innovation to design better programs, policies and practices that improve the lives of Canadians.
Simon Harding is a senior associate at Blueprint. He carries out research across a range of social policy areas that presents clear, actionable insights for policy-makers and practitioners.
Kelly Pasolli is the director of research and innovation at Blueprint. She leads projects and initiatives to support policy-makers and practitioners in leveraging innovation and evidence to increase their impact.
To cite this document:
Myers, Karen, Simon Harding and Kelly Pasolli. 2021. Skills Training That Works: Lessons from Demand-Driven Approaches. IRPP Study 84. Montreal: Institute for Research on Public Policy. DOI: https://doi.org/10.26070/6zwq-8280
The opinions expressed in this study are those of the authors and do not necessarily reflect the views of the IRPP or its Board of Directors.
IRPP Study is a refereed monographic series that is published irregularly throughout the year. Each study is subject to rigorous internal and external peer review for academic soundness and policy relevance.
If you have questions about our publications, please contact irpp@nullirpp.org. If you would like to subscribe to our newsletter, IRPP News, please go to our website, at irpp.org.
Photo: Shutterstock.com
ISSN 1920-9436 (Online)
ISSN 1920-9428 (Print)
Montréal — Au cours de la dernière année, des milliers de Canadiens en âge de travailler ont définitivement perdu leur emploi. Mais une étude de l’Institut de recherche en politiques publiques donne espoir à ces travailleurs qu’une meilleure conception des programmes de formation axée sur les compétences pourraient les aider à retrouver du travail.
Selon cette étude de Karen Myers, Simon Harding et Kelly Pasolli (de l’organisme de recherche Blueprint), les formations axées sur les compétences financées par l’État peuvent être efficaces – contrairement à certaines idées reçues – si elles sont soigneusement alignées sur les besoins des employeurs.
Les chercheurs ont étudié deux modèles de formation largement appliqués aux États-Unis qui ont permis d’améliorer sensiblement la situation des participants, soit la formation sectorielle et l’approche axée sur le cheminement de carrière. Le premier modèle consiste à cerner les besoins des employeurs de secteurs cibles, puis à concevoir des programmes répondant à ces besoins. Le second combine la formation sectorielle à des titres de compétences postsecondaires pour aider les travailleurs à faire carrière dans les secteurs en croissance.
Quatre principaux facteurs expliquent leur efficacité en termes de perspectives et de revenus d’emploi :
« La recherche indique que ces modèles de formation axée sur la demande pourraient jouer au Canada un rôle considérable dans les politiques du marché du travail adoptées en réponse à la pandémie, concluent les auteurs, même si leur mise en œuvre optimale présenterait certaines difficultés. »
Pour mieux aligner nos systèmes de développement des compétences sur les besoins des employeurs, Karen Myers et ses coauteurs recommandent à nos décideurs d’examiner de près l’applicabilité de ces modèles à la situation canadienne. Il leur faudrait ensuite tester et améliorer les plus prometteurs. Enfin, il sera important de créer une solide infrastructure de réseautage entre employeurs et prestataires de formation, tout en produisant des informations actualisées sur le marché du travail.
On peut télécharger l’étude Skills Training That Works: Lessons from Demand-Driven Approaches, de Karen Myers, Simon Harding et Kelly Pasolli, sur le site de l’IRPP (irpp.org/fr/).
L’Institut de recherche en politiques publiques est un organisme canadien indépendant, bilingue et sans but lucratif, basé à Montréal. Pour vous tenir au courant de ses activités, veuillez vous abonner à sa liste d’envoi.
Renseignements : Cléa Desjardins — Tél. : 514 245-2139 — cdesjardins@nullirpp.org