Listen:

Listen on Spotify and Apple podcasts

Episode transcript available below.

Description

What if we’ve been asking the wrong question about program evaluation all along? 

In this episode, pediatric neurosurgeon and medical education researcher Dr. Faizal Haji challenges the familiar “Did it work?” mindset and instead explores a more meaningful question: “How and why do educational programs create change?”. 

Together, we unpack why traditional evaluation methods often miss the bigger picture and discuss practical frameworks that embrace the complexity of real-world learning. Whether you’re an educator, clinician, researcher, or leader, you’ll come away with fresh ways to think about measuring impact, improving programs, and making sense of outcomes that don’t fit neatly into a spreadsheet.

Timestamps

00:37 – What were Ben, Hannah and Justin doing in 2013?

02:35 – An Introduction to Dr. Haji’s paper

05:04 – Ben and Justin share challenges evaluating educational programs

07:59 – Beyond Kirkpatrick: New paradigms in program evaluation

10:08 – Seven essential elements of meaningful program evaluation

11:46 – Hot takes: Complexity science, emergence, and saying goodbye to the pre-post survey

15:31 – Interview with Dr. Faizal Haji

16:10 – From pediatric neurosurgeon to health professions education researcher

18:45 – The origin story of Rethinking Program Evaluation

27:15 – Balancing identities as a neurosurgeon and education researcher

30:00 – Dr. Haji’s current reading recommendations and post-paradigmatic research

31:50 – Closing thoughts and outro

Paper Citation

Haji F, Morin MP, Parker K. Rethinking programme evaluation in health professions education: beyond ‘did it work?’. Med Educ. 2013;47(4):342-351. doi:10.1111/medu.12091

Transcript

00:00:00                    [music plays]

00:00:03 Hannah        Hi, my name is Hannah Kakara Anderson.

00:00:06 Ben             Hi, I’m Ben Kinnear.

00:00:08 Justin           And this is Justin Bullock. And together we are The Backstory Podcast. In health professions education, there is always more to the story, it’s just how far back you are willing to go. We are a medical education podcast that explores literature in health professions education. We’ll provide you with cutting-edge scholarship and cozy conversations with authors. You’ll hear the three of us stumble through the literature, and then we’ll bring on the authors to the podcast for their side of the story. [music fades out]

00:00:38 Hannah        Hello, and welcome to The Backstory Podcast, where behind every paper there’s more to the story. I’m Hannah, and I’m here with mis amigos Ben—

00:00:49 Ben             Hello.

00:00:50 Hannah        —and Justin.

00:00:51 Justin           Nephrology signing on.

00:00:54 Hannah        The paper I’d like to talk about today is almost thirteen years old. And you might wonder why a paper from 2013 deserves to be brought into recent discussion around health professions education scholarship. But I hope after this episode, you’ll have a brand-new appreciation for both this incredible paper and its backstory. So, to get us started, I am going to act a question that may date us a little bit, so this could be risky. But I’d like to know, Justin and Ben: what were you doing back in 2013?

00:01:27 Justin           Well, in 2013, I was still in college. I was a junior. And Ben’s shaking his head. I had not yet fully committed to whether or not I would do medicine. And most of my time was spent—I ran track in college, and so I would say I was thinking about running a lot more than I was thinking about pretty much anything else.

00:01:51 Ben             Back in 2013, I had just finished my Med Peds residency program and was entering my chief resident year. Really focusing mostly on being a new parent—I had a two-year-old—and starting my career as an educator and as a clinician. And I believe I was rocking my iPhone 3. Now I’m on an iPhone 11 and that is very outdated. So if you measure the passage of time in iPhones, we are several iPhone generations from when this paper came out.

00:02:17 Hannah        Yeah, I mean, back in 2013, honestly, I was not in the world of academics by any stretch, and I was very much living in the world of Jersey Shore and keeping up with the cast after the show ended. And yeah, I mean, to be completely honest, I’m still trying to keep up with the cast of Jersey Shore, so maybe time hasn’t changed that much. The paper we’re talking about today is called ‘Rethinking programme evaluation in health professions education: beyond ‘did it work?’’ And I chose this paper for a number of reasons. First, selfishly, it gives me a chance to get on one of my soapboxes, which is that program evaluation is super under-valued and sometimes misunderstood in health professions education. A lot of people want to do curriculum design, a lot of people want to do educational research, but in HPE, very few of us get around to doing really rigorous program evaluation. And that’s a huge problem for our field, as this paper talks about and we’ll discuss it. 00:03:14 Second, when we started this podcast and came up with the idea for getting into the backstories behind HPE, this was actually one of the first papers that I thought of. Partly because I feel like it’s a paper that deserves to be foundational reading—like everybody needs to read this. And partly because I have a feeling that this paper has a really interesting backstory, but you’ll have to wait for part two to hear about that. So, let’s get into the paper. For decades, medical education has been stuck asking the same question about educational programs: did it work? The Kirkpatrick Model has been our baseline standard for guiding program evaluation for over fifty years. Kirkpatrick—if you’re familiar with it, if you’re not—Kirkpatrick is essentially an outcomes-based model that asks educators to consider whether their educational programs—programs here can refer to a curriculum, faculty development, module, workshop, pretty much any educational intervention or structure that you can think of. 00:04:12 It asks educators to measure the success of their programs by collecting data to ensure learner reactions—which is satisfaction in the model—learning, behavior, and results. So, questions like, did students learn? Did behavior change? Did patient care improve? These are all types of outcomes or impacts or effects that an educational program can have. But here’s the problem. It’s actually quite challenging, and it’s often impossible to measure whether a single educational session or intervention or whatever you’ve created actually has an impact on outcomes. Especially multi-layer outcomes like patient care. Not to mention the fact that our educational programs often have a lot of unintended consequences, or outcomes that we just never anticipated. And what do we do when that happens? So, Justin and Ben, have you guys ever struggled to evaluate an educational intervention that you did? Like a faculty development or curriculum?

00:05:13 Justin           I have one from back in the days where DEI was not actively being attacked in the country, where I did a DEI elective in residency. And the purpose was multiple fold. One was to give people sort of intentional scholarly time to actually do Diversity, Equity and Inclusion work, and to attempt to build like a scholarly project, which sort of moved DEI from just like an interest that a lot of us did sort of in our day-to-day work, to something that was part of our like academic kind of repertoire. And I think the reason why it was hard to evaluate is—there were absolutely projects that people did, and somebody would talk on something, or wrote a paper. But there were all of these intangibles that were outcomes of this elective. And part of it is, you know, when we’re—when you’re trying to like write-up something—which we tried to evaluate the elective—you know, you’re looking on a really small-time scale. You know, like two years is nothing in the arc of an institutional sort of shift. And so, you can’t capture things like, you know, first, did the number of residents who were like applying change? Then to the number of people who were actually enrolled. Did the people who come here actually feel safer when they were here? And like these kind of bigger things that, you know, this elective is only one piece in like this sort of compendium of things within an institution. And so, that made it tricky and challenging to think about like what could we attribute to the elective, and like, how do we creatively capture the different ways that an institution might shift with respect to equity and inclusion?

00:06:51 Ben             Yeah, I very similarly have a medical education pathway that I took over from my colleague Eric Warm during my chief year. And same thing, Justin. It’s just so, so many connections and pieces and so much complexity and trying to understand what the outcomes of an education program like that are. And have always really struggled to find a way to evaluate it. And I wish I had found this paper ten years ago when I took over the pathway, because it would have been very helpful. And both for me with that pathway, but also learners and junior faculty that I mentor now around program evaluation projects—it’s so seductive to try to bring our experimentalist brains to the moment and think about like control groups and p-values and things like that. I have since started sharing this paper because I have struggled with this forever. I see people continually struggle with it. And this paper really, I think, frames things in a way that lets you let go a little bit and just try to focus on what’s happening, not, ‘Did it work?’, which is so wonderful. I wish—again, I wish I would have had this paper a decade ago. Which I did, I just didn’t know it.

00:07:58 Hannah        Yeah, you’re not alone, either of you. I mean, both myself, my colleagues, I think all of us in health profession education have had experiences like this, where you’re trying to measure something that feels unmeasurable. And your evaluation brain, like, can’t fully shift out of the experimental or research brain that we’ve been all trained to have. And we’re actually not alone in health professions education either, because this has been a topic of debate outside of health professions education, in the wider field of program evaluation as a whole. Which includes educational evaluation in K through 12 schooling, in non-profit organizations and beyond. And in this paper, Haji and colleagues argue that we need to catch up with what the field of program evaluation figured out about fifty years ago. So, back in the 1960s, education evaluators were doing exactly what we just talked about. Like, running expensive experiments, trying to measure outcomes, and often coming up empty-handed. They realized that focusing only on the question ‘Did it work?’ often misses is the complexity of how educational programs actually functioned in the real world. This paper walks us through several evaluation paradigm shifts that emerged from that realization. Some program evaluation models focus on context and process, asking not just whether you achieved your goals but how you implemented the program and what was happening along the way. These are models like utilization-focused evaluation, and the CIPP Model, or Context, Input, Process, Product. Others emphasized theory, so understanding why a program works by identifying the mechanisms that link the activities of the program to the outcomes. Sometimes this is done by using a formal theory, something like realist evaluation. Some other models embrace emergence, which really focuses on capturing those unintended outcomes and consequences, the adaptations that you do to your program. These things that inevitably happen when you try to implement something in a very messy real-world environment. And ultimately, in order to do program evaluation well, the authors say that we need to ask four questions, not just ‘Did it work?’ First, we need to ask how did it work? Why did it work? And what else happened? To answer these questions, the paper distills seven essential elements that we should evaluate. So both planned or emergent or unintended outcomes, planned and emergent processes, planned and emergent theory, and the overall educational context. And I see two main paradigm shifts represented in this paper. First, thinking of program evaluation not as a one-time snapshot that we take after a program ends. Trying to measure what happened and did it work? And we often do that when we want to write it up and submit it. But the shift here is thinking, instead of that one-time snapshot into an ongoing program of evaluation that runs through the program’s entire life cycle. The second paradigm shift here is thinking way beyond Kirkpatrick for our program evaluation models. I think depending on the kind of program that we’re running and the context that we’re working in, we really ought to consider models from program evaluation outside of health professions education. Models like CIPP, others like developmental evaluation, which is a new one for me, that I’m learning about. And we can look into theories like realist evaluation. Okay, now that I’ve covered the meat of this paper—and I highly suggest that you read it outside of this episode—I’d like to know, Ben and Justin, you read this paper. And I know you’ve got some opinions about it, so it’s time for your hot takes.

00:11:54 Ben             Well, I love this paper. One of my favorite things about it is how it wraps in complexity science, which is not novel in health professions education. People have been thinking and writing about this for a long time. But this was my on-ramp, and I’ve since fallen down a rabbit hole of learning about complexity science and the different ways it can help us see what we’re doing. My, I think, hot take is this approach that Haji and colleagues advocate for kind of pulls us to not just focus on the narrow question, ‘Does it work?’, and to think more openly and creatively and curiously about what’s happening, and understanding the world. I wonder how much we can do that in a world where so much of our work is funded, and often with funding comes the need to demonstrate outcomes. And can we use this paper and discourse around this paper to nudge our field away from having to connect a return on investment every bit of programming that we do?

00:12:50 Justin           I have—I’m going to call it more of a hot ramble, not a single hot take. [chuckles] At my institution I have a role where I basically support faculty in education scholarship. My brain is a very simple place and I tend to like have these bit buckets where I put things. If the goal of research is to prove, the role of QI is to improve. And I put program evaluation somewhere in between the two. It’s kind of doing both simultaneously. Historically my program evaluation bucket was completely limited to CIPP model—Hannah, you’ve already mentioned—and then the realist, like, did it work, for whom, in what contexts? And I love this idea of like emergence and centering emergence, which basically is like… you know, you come in with this idea and maybe you’re implementing something that’s, like, very commonly done. Let’s say you’re implementing a point of care ultrasound curriculum. And at this point, you know, there are many, many institutions who have done this. But there may be certain things that emerge as you’re implementing this curriculum. There may be certain ways in which implementing them in my unique context may lead to some important insights that have not previously been captured. Without thinking about this topic of emergence as a core component of program evaluation, you miss the learning that happens along the way that you didn’t necessarily start with when you began your program.

00:14:10 Hannah        My hot take, honestly, is that I think we’re done with the pre-post survey. I think it’s time we move on. It’s been—it’s been time to move on. And, you know, demonstrating change in a pre-post survey is interesting but it’s not nearly the most interesting thing about a program. So, I’m ready to say it’s time to put the pre-post survey out to pasture.

00:14:33 Justin           That is a hot take.

00:14:34 Ben             I’m totally down with that. And as a side note, Justin, if you’re ever a fighter pilot, I think hot rambles should be the name that you put on your helmet. That was a great ramble. You’re great at hot rambles, and it’s also a great nickname.

00:14:45 Hannah        Justin ‘Hot Ramble’ Bullock. I like it.

00:14:48 Justin           I’m getting a tattoo.

00:14:49 Hannah        By the way, Justin and I have a tattoo pact. So, this is—this is real, you better ask us about it later.

00:14:55 Justin           I was going to say, are we going to announce it to the whole world?

00:14:58 Hannah        I’m not saying this has to make it on to the final episode. I’m just saying, you know.

00:15:02 Ben             This is perfect because if you were to evaluate the impact of this podcast, matching tattoos is an emergent phenomenon.

00:15:09 Justin           [chuckles]

00:15:09 Hannah        [chuckles]

00:15:09 Ben             I don’t think that would be in the program theory of what this podcast is supposed to do. But if you both end up with a ‘Hot Ramble’ tattoo on your forearm, that’s an emergent outcome.

00:15:19 Hannah        Yeah, that’s a great example, Ben. I love it. Alright, with that we’ll close part one. [laughs] I’ll hand it to you, Ben. Take over for part two.

00:15:32 Ben             So, we are so lucky to be joined by Dr Faizal Haji, who is a pediatric neurosurgeon and the new director for the Centre for Health Education Scholarship, otherwise known as CHES, at the University of British Colombia, otherwise known as UBC. Dr Haji, welcome to The Backstory Podcast and thank you for joining us. How do you prefer that we call you?

00:15:54 Faizal           Faizal is great.

00:15:56 Ben             Fantastic, thanks, Faizal. Could you tell us a little bit about who you are, what you do, and something that the listeners might not know about you?

00:16:04 Faizal           Sure. Well, I think you kind of gave the sort of headline titles already. So, I’m Faizal Haji. I’m a pediatric neurosurgeon in my clinical life. I work at the BC children’s hospital. And I also just took over as the director of this amazing education research center at UBC, called CHES, which is a place that I have long wanted to work when I was in training, and I feel like it’s a dream come true to finally be here and to be in a leadership role in the center. So, it’s been great for me. I hope that mt engagement with the center is helping move things along in the HPE, health professions education, both locally and elsewhere. And so, I think that will be a wait and see kind of thing. But I have been engaged in research in health professions education for… I think now going on thirteen, fourteen years. I started my PhD partway through my residency. So I did my residency at University of Western Ontario—or Western University I think it’s now called—in London, Ontario. And that was in 2008. And then 2011 I started my PhD at the Wilson Centre and this paper was actually one of the things that came out of the process around me learning about health professions education, and kind of being enculturated into the space. So we could talk a little bit about that. And then that was what really started my journey into health professions education scholarship, and I’ve been involved in research there. I had a brief stint where I was in the US doing a fellowship in pediatric neurosurgery in Alabama, and then came back to Canada and worked in—I was a medical education scholar and neurosurgeon at Queens University for a couple of years, and then came up to UBC and have been here since. Most of the work that I do now has actually moved away from program evaluation and it started in simulation and instructional design, and that’s kind of what the focus of my research was during my PhD. Now I’m far more interested in lots of socio-cultural topics, and particularly in the context of surgical education, because it’s stuff that emerges as a clinician scientist from the day-to-day things that have been, in my own experience and the experience of my colleagues and the experience of my students, the experience of my patients in trying to navigate the education and healthcare systems that are inextricably intertwined. And so, lots of interesting stuff around clinical reasoning, the role that basic sciences play in the way that we make decisions. Entrustment and thinking about what that means from the learner perspective. Transitions into practice and how we manage the roles as educators and learners. And faculty, and most responsible physician when you’re in a faculty role, some interesting stuff around neurodiversity that’s coming up that I’m really excited about. Yeah, lots of different things.

00:18:45 Hannah        Very cool. So, I’m going to ask you next about sort of where you are now. But I am interested about the experience of writing this paper. You mentioned you did it at the Wilson Centre when you were sort of getting enculturated into HPE. What was the experience like of writing this paper, you know, as a person new to HPE trying to get into the field?

00:19:08 Faizal           I think the best description I can give for it, it was a labor of love. And honestly, it was a really fantastic experience writing the paper. And I can say that both on a personal and professional level. I think it’s still my most highly cited paper, which is pretty crazy [chuckles] considering it was I think either the first or second paper that I wrote in health professions education. So, I really didn’t know what I was doing when I did it. And the story is interesting because it came out of—I was—when I was at the Wilson Centre, I was taking a course on research methods in HPE that was being run by Tina Martimianakis and Maria Mylopoulos, who were two researchers at UFT and at the Wilson Centre. And one of the assignments for the course was to write a paper about something to do with HPE. And so, I picked program evaluation because I didn’t know a lot about it and I wanted to learn about it, and I thought this is something that might be useful for me. Because at the time, I didn’t really know if I was going to do research, or if I was going to do educational leadership, or curriculum design, or program evaluation, or kind of what my space was going to be. And so, I got involved in a group, which was the co-authors on the paper—Marie-Paule Morin and Kathryn Parker. So, Kathryn was a pretty high-level program evaluation expert at University of Toronto and she’s now at Holland Bloorview, which is the kids rehabilitation hospital in Toronto. And she was sort of the local expert in program evaluation and taught in the course that we were taking. And Marie-Paule was a colleague of mine who was doing her PhD in the same lab that I was in. And she and Kathryn had actually started to put together this paper, and were thinking about what to do with it, and they’d kind of gotten stuck and weren’t sure where they wanted to take it. And I needed to do an assignment [chuckles] for this program, for this course, and so I turned to Marie-Paule and I said, “Can I—can I like dig into this and write something and let’s see where it goes?” And so, the first draft of the paper was really this assignment that I submitted. And I think it was supposed to be like twenty-five hundred or three thousand words or something like that, like a course paper. And I submitted something to Tina that was six thousand words because it was just me like—just mind-dumping onto this page with all these things that I thought. And she was the first person that turned to me and said, “This is good. Like, you could actually turn this into a paper if you wanted to. There’s the nuggets of what’s a good paper in here, but it’s like crazy long, and it’s rambly, and there’s too many things. [chuckles] And so, you’ve got to pare it down to something that’s meaningful.” And so, the actual writing was a lot of me and Marie-Paule and Kathryn sitting down and just like trimming and editing, and figuring out what the narrative was going to be, and what was actually important to say, and what was stuff that people could figure out on their own. What did we actually want to communicate? And so it was a really—for me, a really developmental process in how to write and how to write well, and how to think through writing. And how to communicate something that I was learning that I thought could be useful for other people to learn too.

00:21:59 Hannah        So, in your day job now—I mean, you are a full-time program evaluator. You’re a busy clinician, you’re an education leader. What did this paper teach you that you sort of carry with you, even as you’ve expanded your scholarship portfolio beyond program evaluation?

00:22:17 Faizal           It’s challenging to think about always just trying to have to prove that this thing that you’re doing is valuable and is meaningful. Because that’s really what a lot of people go to when they think about evaluation, is, “I’ve got to show somebody that this did something that makes it worth continuing to invest in.” And so, that puts a pretty high bar on whatever you’re doing to basically prove that it works. And I think the whole goal of the paper was that you can let go of some of that. That it’s not always about proving merit or worth. One of the core readings that’s embedded in the paper and that people can go look at is… there’s this thing called the evolutionary tree of program evaluation. So it’s, as the field of program evaluation and K to 12 education evolved, a bunch of theorists who were writing about the process of how to do program evaluation and how to think about program evaluation kind of ended up in these different branches. And there’s this beautiful metaphor of a tree and three different branches. And there’s the use branch, there’s a justification branch and there’s a theory branch. And so, it just spoke to this idea that why you’re doing the evaluation can have different purposes, and you might pick a particular approach to evaluation that focuses on one of those things or other things, and what would that mean for you in terms of what you’re trying to achieve? And so, if your program evaluation is really about generating knowledge, then you might be in the theory branch because you might be thinking about how can I generate transferable knowledge that helps us understand how these programs or these functions work? And how can I disseminate that in a way that is meaningful for other people to take up and do things with and improve our educational processes at large? And whereas if you’re really focused on how do I make sure that this is the best program locally and how do I improve my program iteratively, then you’re going to be in that use branch, and it’s really about engaging the stakeholders and figuring out what data is useful and informative, and like, what processes are working for you, what processes aren’t working for you? And then if you’re somebody who is at a high-level funding kind of decision-making table, and you really need to know, “Is it worth us continuing to invest our time and money and effort into these activities?”, then you are in that justification branch and there is important outcome-related data that you’re going to need to generate. So, it’s not about one thing being more important than the other. It’s about really understanding what’s the goal that you’re trying to achieve. I think actually the people that have come after we wrote this paper and who have reflected on it have done a really good job. So Betty Onyura has written some papers about program evaluation that I would encourage people to go look at. One of which was talking about ‘What information, for whom?’ That’s not the exact title but it’s something like that. Where she talks about how different evaluations can have different purposes and how you sort of negotiate those purposes among stakeholders and people who are involved in the program evaluation, will ultimately determine what you get. And so, thinking through that and having conversations about that is important. And so, I think that’s sort of an add-on piece that I’ve taken away.

00:25:12 Justin           I have one comment about something that you said earlier, which is, you know, this is your most cited paper even though it was one of the first papers that you wrote. Someone once told me that when you look at most sort of intellectual greats through history, most of them made their most significant contributions at the very beginning of their time in the field. And the theory is that it’s largely because people have not yet been enculturated to the norms of their profession, and for that reason, they like think differently than the people who are already there, you know. They—it’s—it’s far enough in that they develop some skills in the areas—they may not be, sort of have the deepest skill expertise, but they think in a different way.

00:25:52 Faizal           That makes sense. I think that’s probably true. I think in many ways when you’re a graduate student, there’s two really great things that happen. Which is one, you are—you have the time and space to just be free to explore and figure out what you want to spend your intellectual resources on. And I think that’s kind of what happened with this paper is that, yeah, there was a course paper that I had to write, but then I really got into it and I was like, oh, I have the time and space and I can really invest the time to make this thing something that I was proud of. And that my co-authors and I were all proud of, because it was definitely a massive group effort. I think the other part of it is what you just said, which is that you’re not encumbered by all the different things that you’ve been enculturated to think and you do have the opportunity to think differently, and I think that’s a really nice space to be in. Justin, you’ve done some of this stuff too, in the ways that you’ve written about various topics and flipping things on their heads. People make associations about how even methodologically you have to do things, and you’ve come along and said, “I think you can put critical theory and quantitative research together. I think there’s a way to do that.” Maybe it’s because naturally you’re inclined to think differently by virtue of the paradigm that you exist in or that you naturally gravitate towards.

00:27:07 Justin           Well, I call it just complete ignorance. That’s my—you just, [chuckles] you know, stumble forward and [chuckles] there you go. Related to this, I wanted to hear you talk about how you sort of switch back and forth between the different sort of identities that you exist in. You know, as a neurosurgeon, I imagine that brings a very specific set of, like, thoughts and thinking patterns and beliefs about reality and truth. And then as a researcher, I wonder whether you exist in a different sort of space?

00:27:41 Faizal           Yeah, that’s a really interesting question. I think yes, I probably do. I think that I’ve tried a lot to bring both worlds together in a way that is productive. First, on the neurosurgery side of things, yeah, anybody who is a practicing clinician will know that there are biomedical ways in which we think about medicine and healthcare and the practice that we engage in, and a lot of the decision-making and things are based on what’s the right thing to do in this moment and how do I manage this case? And a lot of the teaching is around that. But then there’s also a lot of human elements in it too, which I think my research has enriched. So, now I think a lot more about what is it like for the gender diverse trainees that are in my program? So, we have an Indigenous resident who is in our program and I’ve thought a lot recently about what it means for her to be in our spaces, and bringing herself, and her whole self, to her training. It’s forced me to think a little bit differently about my interactions with patients and with families. I sort of alluded to the idea that our education and healthcare systems are inextricably intertwined, and I think I appreciate that a lot more now with the researcher hat that I have, and—but it does require translation in a way that is challenging sometimes. I do think there’s a space for us to think differently. Sometimes I’m the contrarian in the group, and whether it’s on our grand rounds or whether it’s in our conversations where I’m saying, “Well, what about this, or how do we think about that?” And I’ve been pleasantly surprised that it’s been received actually pretty well. On the research side, I think the real benefit of being a clinician is that you bring a practical orientation to the work that you do. I naturally gravitate towards questions that are focused on the educational spaces in which I am operating. And so they’re imminently practical. But I think the research side of me has evolved a lot from being very post-positivist, based on my biomedical training, into somebody who really understands that the world, particularly the social world, is not something that is all X’s and O’s, and has direct truth. And that we do need to challenge a lot of our assumptions about what we understand and the discourses that we operate in, and all of those kinds of things.

00:30:00 Justin           For our final question, we would love for you to recommend something that you’re either currently reading or recently reading that was impactful to you.

00:30:11 Faizal           I have to be honest that I haven’t been able to do a lot of reading for myself and for fun. It’s—a lot of my reading has been kind of operational, instrumental for the purposes of achieving something that I’m working on. But there’s a couple of pieces that I’ve bene thinking a lot about that do relate to some stuff that I’d like to write and haven’t gotten around to. And so, it’s not stuff that I’ve read but it’s stuff that’s on my bucket list. The first was a piece that was shared from one of my colleagues Rola Ajjawi, who’s… here at UBC and is also the editor-in-chief now of Medical Education. And I talking to her about how it’s important to have kind of an epistemic voice and a particular philosophical orientation to the work that you do, and be clear about that and talk about that. But that when you zoom out and think about the sort of really complex, troublesome problems that we deal with in health professions education—those are things that I don’t think you can just understand from a single perspective. You need a diversity of ways of looking at these challenging problems in order to really get to them. And then—so, Rola said, “Well, I think what you’re talking about is post-paradigmatic research.” So, some of the work that’s been done in social emotions research that focus on post-paradigmatic research, I think is actually the kind of stuff I want to get to. So, the stuff that she’d sent me was Rebecca Olson’s work in ‘Post-paradigmatic approach to analysing emotions in social life’. And Epistemic Fluency and Professional Education, which is a book that is by Goodyear and Markauskaite. So that’s actually the next thing up on my list of things to read, because I feel like I want to my head around what that means to be operating in a space that goes beyond our traditional paradigms.

00:31:52 Justin           Thanks very much. Once again we want to thank you, Dr Faizal Haji, for coming onto The Backstory Podcast. [music fades in] We’ve been talking about the paper ‘Rethinking programme evaluation in health professions education: beyond ‘did it work?’’ This is in Medical Education circa 2013. Thank you again listening to The Backstory Podcast, where there’s always more to the story, it’s just how far back you’re willing to go. Talk to you soon.

00:32:20 Hannah        The Backstory Podcast is supported in part by the Josiah Macy Jr Foundation. Production and editing by Rachel Holloway. Music by Hannah Kakara Anderson. Artwork by Kaitlyn Ripaldi. The Backstory Podcast is a sponsored project of the Learning Design Collective. Thanks for listening. Just a reminder that the opinions expressed here are our own, and don’t represent the views of any institutions. This podcast is for informational and entertainment purposes only. For show notes and links, visit our website. If you enjoyed this episode please subscribe and leave us a review, and check us out on LinkedIn, Instagram and Bluesky, @TheBackstoryPod. [music fades out]

[End of recording]