Disentangling the Meaning of Multisource Feedback: An Examination of the Nomological Network Surrounding Source and Dimension Factors

This study combines internal and external approaches to construct validation in examining the construct validity of multisource feedback (MSF). First, consistent with prior MSF research, within source agreement was greater than across source agreement, the MSF instrument was equivalent across sources, and source and dimension latent factors characterized the MSF data. Next, existing MSF construct validity research was extended by examining the pattern of relationships between factor analytically derived source and dimension factors and externally measured constructs (e.g., assessment center dimensions, personality constructs, and intelligence). The pattern of relationships between MSF dimensions and conceptually similar and dissimilar external constructs suggested somewhat weak construct validity evidence for the MSF dimensions. In contrast, based on correlations with externally measured constructs, relatively strong evidence was provided for the construct validity of AC dimensions. Finally, the MSF source factors were differentially correlated with externally measured constructs, suggesting that MSF source effects represent substantively meaningful source specific variance, as opposed to error. These findings are discussed in the context of managerial skill diagnosis and the efficacy of collecting performance data from multiple sources.