Agreement in classification choices by school psychologists when evaluating children for special education placement

A lack of diagnostic reliability among school and clinical psychologists has plagued the field for years. Diagnostic agreement rarely exceeds chance by more than a few percentage points. This study examined the level of agreement when school psychologists studying only objective data attempted to classify test profiles of children referred for special services.

Sixty Tennessee school psychologists each examined four test profiles. Diagnostic data were limited to scores from the Weschler Intelligence Scales for Children-Revised, the Wide Range Achievement Test, the Bender Gestalt, the Walker Behavior Checklist, and the Vineland Social Maturity Scales. Age and grade levels were included. Diagnostic criteria used were from the Tennessee Student Evaluation Manual.

Of the sixteen profiles four had an original diagnosis of Educable Mentally Retarded, four Learning Disabled, four Emotionally/Behaviorally Disturbed, and four had been designated not eligible for special services. School psychologists received four sets of test data randomly chosen from the 16 sets, and were asked to classify each set as belonging to one of the four categories. Demographic data were also gathered. Three aspects of the study were examined, overall diagnostic agreement, consistency with original diagnosis, and agreement on EMR diagnoses compared to LD, E/BD, and NE.

The overall percentage of agreement on diagnosis was 72%. Agreement on all cases exceeded chance (25%) and all but one case was significantly different from chance using a chi-square test of significance. When matching sets of data to the originally diagnosed categories there was 74% agreement in the LD category, 73% for EMR and NE, but only 8% for the E/BD category.

The results of the study provided evidence that objective data and uniform diagnostic criteria may enhance the probability of diagnostic reliability in case of EMR, LD, and NE diagnoses. The lack of reliability obtained on E/BD cases indicates that diagnosis in this category cannot be made consistently based on the objective data provided.