Use of the critical incident method in developing an instrument for evaluating students in medical laboratory programs

Evaluation of the clinical performance of students is extremely important in all areas of the health field. The purpose of this study was to develop an instrument for evaluating students in the clinical portion of medical laboratory training. The reliability and validity of this instrument were established.

Data for the development of the instrument was obtained through response to mailed inquiry sent to 350 Program Directors/Educational Coordinators of Medical Technologist (MT), Medical Laboratory Technician (MLT)/Clinical Laboratory Assistant (CLA) Programs accredited by the American Medical Association. Thirty-four percent responded. These represented 37 states. The developed instrument was then utilized in five programs (two MT Programs, two MLT Programs and one CLA Program). The Program Director/Educational Coordinator and one instructor of each of the participating programs provided additional information relative to reliability and validity of the instrument.

The data for the development of the instrument was treated according to the approach recommended by Herbert and Attridge. Coefficients of observer agreement were calculated using Scott's Pi. Factor analysis was used to determine suitability of categories used. A least square regression analysis was conducted to test the degree of relationship between subjective instructor evaluation and ratio of effective to ineffective observed incidents recommended by Whitlock for summative evaluation. It was found that the relationship of instructor rating and effective/ineffective ratio is based more on an agreement of importance and correct effective/ineffective classification than on the exact categorizing of the observed incidents. Further, the recording of effective and ineffective behavior of students during clinical practice could be used in an objective manner to assign grades.

Selected recommendations from the study were:

1. The present study should be refined and replicated in other selected vocational areas which have an internship, externship, practicum or co-op as an integral component of their educational program.

2. The perceptions of clinical faculty regarding their dual roles as staff technologists and educators should be statistically analyzed. The clinical/faculty member is the key person in clinical evaluation of students.