Development of the Biostatistics and Clinical Epidemiology Skills Assessment for Medical Residents

This study developed the Biostatics and Clinical Epidemiology Skills (BACES) assessment, and established its preliminary item characteristics and validity evidence. Unlike previous instruments, the BACES assessment was developed and tested using an item response theory (IRT) approach to measurement to create a new, adaptive biostatistics and clinical epidemiology knowledge assessment for graduate medical professionals. Thirty multiple-choice questions were written to focus on interpreting relevant examples of clinical epidemiology and statistical methods. A four person expert panel reviewed these items for content validity. After this review, the BACES assessment was administered to 147 medical residents across three academic medical centers. Results of the IRT analysis produced a final instrument of 26 items with 13 devoted to statistical methods and 13 to clinical epidemiology, which successfully fit a 2-parameter IRT model. In contrast to previous assessment research, an IRT approach allowed for each BACES item’s difficulty, discrimination, and reliability to be estimated separately from the sample on which it was tested. As a result, this preliminary study has paved the road for a flexible yet psychometrically rigorous instrument for measuring the biostatistical and clinical epidemiologic knowledge of graduate medical students.