Behaviorally oriented content validity and utility of a training program

A second phase of the study examined the relationship between the content validity and criterion-related validity. of the training program. The hypothesis predicted higher validity coefficients for behavioral dimensions with high "overlap" between training and job behaviors. That is, greater correlations were expected for rating dimensions which had been established as more content valid. A performance rating scale, comprised of the 10 "overall" PAQ dimensions, was used for ratings of performance during training of 32 police recruits by the three training officers who analysed the training program. Six months later, each recruit's three most immediate supervisors rated their performance on the job with the same rating scale. Results showed there was no relationship between the PAQ-derived indices of content validity and corresponding empirically derived estimates of criterion-related validity. Results, however, may have been due to restriction in the range of either or both sets of validity indices. The PAQ-generated data suggested that restriction occurred in the range of the distribution of absolute difference values (the indices of content validity), and that the range was restricted to values at the lower end of the distribution (i.e., high overlap condition). Under the condition of high overlap, the hypothesis predicted corresponding significant empirically derived validity coefficients. The fact that 8 of 10 criterion-related estimates were significant is consistent with that hypothesis. More research is needed to evaluate the hypothesis under the full range of overlap.