In May, the IRPP published Harnessing Generative AI: Navigating the Transformative Impact on Canada’s Labour Market, a study by Matthias Oschinski and Ruhani Walia looking at the automatability of Canadian occupations. They employ an innovative methodology, using large language models (LLMs) as stand-ins for human experts to rate the extent to which generative AI tools could master different skills and work activities.
I suspect this technique could be very valuable for a wide range of researchers and analysts, inside and outside of government. As it turns out, the core task — systematically prompting an LLM with a list of items to get a structured response (like a rating between 1-5) — is surprisingly simple to implement with modern tools.
Of course, the question is not just whether we can use an LLM, but whether we should. As a new methodology using emerging and sometimes unreliable tools, results should be approached with caution. At the same time, the only way to learn is to experiment.
To that end, I replicated the first part of Oschinski and Walia’s study (producing AI-generated ratings for skills and work activities) and published it for others to use and adapt. You can access it using the link below.
My hope is that this project encourages other researchers to experiment with this new approach to labour market analysis.
You can use the provided code to:
I am grateful to Matthias Oschinski and Ruhani Walia for developing the original methodology, for their permission to replicate their study, and for their valuable comments on a draft of this guide.
The views expressed here, and any errors in the accompanying notebook, are my own. The notebook was developed with assistance from Gemini (2.5 Pro Preview) while learning the DSPy framework. I take full responsibility for its content and accuracy.
