The Effect of a Screening Instrument to Determine Teacher Effectiveness as Measured by Academic Gain Scores

Schools and school districts are constantly faced with the task of screening and hiring potential teachers who will impact student achievement. The Knox County School System addressed this challenge by using a screening instrument called the Urban Teacher Perceiver, which was designed by The Gallup Organization. Tennessee used a statistical methodology entitled the Tennessee Value Added Assessment System to measure teacher effectiveness. The purpose of this study was to determine if there was any effect between a selected teacher-screening instrument and teacher effectiveness as measured by academic gain scores in mathematics, reading, and language. Also, the study examined the interaction effect between the screening scores and gender, race, age, when hired, grade level taught, and school setting (urban, suburban, or rural) in regard to teacher effectiveness as measured by academic gain scores in mathematics, reading, and language.

The population consisted of two hundred forty-two third through eighth grade teachers in the Knox County School System who were hired between 1993 and 2001 and had both an Urban Teacher Perceiver (UTP) score and a three-year average Tennessee Value Added Assessment System (TVAAS) score in mathematics, reading, or language. The sample also included demographic information: gender, race, age when hired, grade level taught, and school setting. The data were analyzed using an ANOVA to determine if there was a main effect between the UTP and TVAAS scores for teachers. The other five hypotheses used the General Linear Model procedure to investigate the interaction effect between all of the factors.

The data analyses were generated with a statistical significance set a .05 level. The six hypotheses investigated in this study revealed that there was no significant effect in any of the factors statistically assessed. The effect between a teachers’ screening score on the Urban Teacher Perceiver (UTP) and the effectiveness of their teaching as measured by the Tennessee Value Added Assessment System (TVAAS) demonstrated no significant main effect in mathematics, reading, and language scores. Also, there was no significant difference in the interaction effect between the overall UTP score and gender, race, age when hired, grade level taught, and school setting in regard to TVAAS scores in mathematics, reading, and language, except for the interaction of the UTP and particular grade levels in regard to mathematics.

The conclusion of the study was that the teacher’s overall Urban Teacher Perceiver score does not provide adequate information to predict a teacher’s effectiveness as measured by academic gain scores. The school system was recommended to re-evaluate their reasoning for the use of the screening instrument and not to use it as the sole determiner for hiring potential teachers.