Masters Theses

Jianming Shen

Date of Award

12-2001

Degree Type

Thesis

Degree Name

Master of Science

Major

Industrial Engineering

Major Professor

Garry D. Coleman

Committee Members

Denise Jackson, Max Hailey, Bruce Whitehead

Abstract

This study measured the rating effectiveness (interrater reliability, halo error, leniency or severity, range restriction) for each sector (Private Manufacturing Sector, PrivateService Sector, Public Local Sector, and Public State and Federal Sector) of a StateQuality Award using examiner scores from the 1999 award process. Results were compared with the rating effectiveness of the 1998 State Quality Award examiner scores.Interrater reliability for the Private Manufacturing Sector and range restriction for all sectors were recalculated for the 1998 State Quality Award examiner scores.The Private Manufacturing Sector's scores exhibited,low to moderate interrater reliability in both 1998 and 1999. Both the Private Service Sectors and the Public LocalSector's scores exhibited low interrater reliability in 1999, which is lower the the 1998 slow to moderate interrater reliability. The Public State and Federal Sector's scores exhibited low interrater reliability in both 1998 and 1999. Halo error was present for all examiners (both years) whose halo error could be measured by the determinant of the correlation coefficient matrix, Scores from most of the dimensions in each sector showed severity in the 1999 process, while scores from most of the dimensions in each sector showed leniency in the 1998 scoring process. Scores from only a few of the dimensions in each sector showed range restriction for both the 1998 and 1999 scoring processes.Due to the small sample size, caution should be taken in generalizing the results of this study. The small sample is a result of using field data from an actual Award rather than a controlled laboratory setting.

Download
Files over 3MB may be slow to open. For best results, right-click and select "save as..."

Share

COinS