A Realist Evaluation Approach to the Study of a K-12 Online Professional Development Course

The primary purpose of this study is to determine the extent to which the desired outcomes for select K-12 teachers' participation in a specific arts-based online professional development (OPD) course were being achieved, for which teachers (or subgroups of teachers), in what types of contexts, and how. A secondary purpose of the study is to set a precedent for future studies of K-12 OPD courses, particularly those using a realist evaluation (RE) approach.The RE approach develops, tests, and refines a set of program theories, or hypotheses, about what outcomes will be achieved, for whom, and in what circumstances (Pawson & Tilley, 1997). In accordance with this approach, I began by developing initial program theories that hypothesized the extent to which the desired course outcomes were being achieved, for which teachers (or subgroups of teachers), in what types of contexts, and how. I then used multiple forms of data collection and analysis to test and refine these program theories during three iterative phases: Develop Initial Program Theories, Test and Revise Program Theories, and Refine and Finalize Program Theories. As RE approaches are retroductive, I analyzed the data as it was being collected. I gathered data from course completion rates, a realist review of the literature, interviews, course annotations, and responses to question and reflection prompts embedded in the course. Collectively, these data were used to determine the extent to which desired course outcomes were being achieved, for which teachers, in what types of contexts, and how. Ultimately, six program theories were finalized, from which five middle-range theories were derived. These findings identified relationships between the contextual factors, causal mechanisms, and outcomes from the VTSB course. They indicated that teachers engaged in, applied, and shared course content when the content aligned with their pedagogical beliefs, when they experienced pedagogical discontentment, and when they had strong interpersonal relationships with school leadership. Findings also indicated that teachers' engagement in the course was constrained by mandated curricular requirements, time demands, an over-focus on accountability measures, and when course examples were not representative of their diverse students and instructional contexts.