Developing a job analysis based performance appraisal system at the United States Military Academy : a new approach to forced-choice evaluation
The objective of this study was to apply new techniques in developing a job analysis based, multidimension forced-choice appraisal instrument at the United States Military Academy. The Position Analysis Questionnaire (System II) was used to identify job families and key job family performance dimensions in a 19 job sample. Forced-choice items were generated and weighted as to performance dimension importance. The completed five dimension performance report was then field tested within six cadet companies using job holder samples of N=53, N=64, and N=57 in the process. The psychometric properties of the experimental scales were superior to other scales when members of the cadet chain of command were rating subordinate performance. When the company tactical officer was rating, the new scales performed as well as any other scale tested. Multidimension-multirater analysis indicated that three of the five experimental scales possessed convergent validity and some discriminant validity. When members of the cadet chain of command rated the performance of subordinates with the new scales, the performance dimension scores predicted ratee scores on four other performance criteria with validity coefficients of .35, .44, .45, and .55 respectively. When company tactical officers rated with the new scales, the performance dimension scores predicted ratee scores on five other performance criteria with validity coefficients of .45, .54, .57, .75, and .77, respectively. When an overall performance rating was combined with the dimension scales, the performance of the combined scales was superior to either one alone.
Thesis78b.M475.pdf
7.6 MB
Adobe PDF
7e323fbe600f3b77282adc653595255f