Temporal stability : a comparison of four estimating procedures

This study examined the relative merits of four methods to assess the temporal stability of rank order: test-retest correlation, generalizability theory, covariance structure analysis, and correspondence analysis. In addition, this study investigated the stability of construct relations among job involvement (JI), job satisfaction (JS), and life satisfaction (LS) by using the LISREL model with multi-sample analysis. Data used for this study were drawn from a study by Lounsbury and Hoopes (1988). The measures (JI, JS, & LS) in their data were obtained from a large survey of work and nonwork relations. This survey took place in two phases, conducted five years apart. The following conclusions were formulated based upon the findings of this investigation.

1. Each method has adequate theoretical rationales for assessing temporal stability.

2. Theoretically, covariance structure analysis via the LISREL model has at least three advantages when assessing temporal stability: (a) adjusting for errors of measurement and correlated errors of measurement, (b) dealing with observed and unobserved variables, and (c) testing the measurement model under study.

3. Covariance structure analysis yielded more stable outcomes with small samples (n = 20) than other methods. However, it sometimes failed to estimate coefficients of temporal stability.

4. Overall, covariance structure analysis produced more stable outcomes with moderately nonnormal data. Results of LISREL approaches with an ordinal scaling (e.g., using a polychoric correlation matrix) were identical when compared with the results of the original data.

5. On the bases of fit indexes (CFI & GFI), the construct relations of Time 1 were equivalent to Time 2 in terms of construct loadings and correlations. However, the χ² difference tests revealed that the model representing the same constructs over time was a best model. With the information obtained from this investigation, covariance structure analysis appeared to be a promising method for studying temporal stability. However, generalizability theory and correspondence analysis provided additional information to assess temporal stability. Generalizability theory identified sources of measurement error and correspondence analysis produced graphical displays of relationships among variables or categories. Therefore, a combined approach, covariance structure analysis along with generalizability theory or correspondence analysis, will provide researchers with a more complete understanding of temporal stability.